Hmmm, this isn't a trivial problem.
Obviously all our code runs as part of page loads, subject to the main php.ini settings. The way we run cron plugins is by hooking in to Joomla's 'onLastProcess' event (or whatever it's called), which allows system plugins to be run as the very last thing during a page load. And doing CSV imports is a time consuming operation, as there's a lot of processing we have to do, which can easily exceed reasonable max execution times set in php.ini designed for normal web usage.
Delaying the inserts wouldn't buy much, as most of the overhead is in our internal processing rather than the actual database operations, working out what needs to be inserted, and what should update existing rows, and whether it's raw values, and whether there are any new elements to add, and whether there is joined data we need to handle, etc etc. We also couldn't do delayed inserts if there is any joined data (due to the chicken and egg problem of updating the FK with the newly created PK val of the joined data).
The only way I can think of is essentially what Rob is suggesting, but it would need a little "wrapper" script which instantiates the Joomla framework, and as much of Fabrik as is needed to directly call the cron CSV import plugin.
One other option might be for us to add another option to the generic cron plugin params, where you could specify a PHP script timeout value, and we could attempt to do an ini_set() for max_execution_time before running the plugin. This would be an easier quick fix.
As an experiment, can you try this:
In ./plugins/fabrik_cron/importcsv/importcsv.php, around line 70, so it's the first line in the process() method, put this:
PHP:
@ini_set('max_execution_time', 300);
... and try running your import task using the "Run" button on the Scheduled Tasks page.
300 (5 minutes) should be enough for fastserv, with 7000 rows.
For sales2010, you may need more than 300, for your 100k rows. I have no idea how long it would take for that many rows, although you could exptrapolate by testing with 1000 rows, see how long it takes, then multiplying by 100 (and add about 10% for luck).
NOTE - this will only work if your host allows setting of INI variables in the code, which typically is not the case in cheap shared hosting plans, but should be possible with any kind of virtualized or dedicated plan. You'll be able to tell if the ini_set() worked, by timing how long it is before the script times out, and seeing if it was the value set in php.ini, or the value you used in the ini_set().
-- hugh
