Slow site

::sigh::

I have no idea why, but J!'s debug info (profiling, etc) is not showing on that test site (No 2), even with J! system debugging and the debug plugin enabled.

Can you resolve that and let me know when it's working?

-- hugh
 
Oh, you'd set access to "Guest" for the debug plugin. Which means it ONLY shows for guests, not logged in users. I've set it to "Special".

-- hugh
 
Hmmm, well, I'll try, but I'm really not sure I can do much to speed this up, with 75+ calc elements in the form. We really did not design the calc element with that in mind! I think I had "half a dozen or so" as the upper bound of likely usage when I wrote that code originally.

The biggest chunk of the time is going in two places. Validation (about 4 seconds), and the email plugins (about 6 seconds).

Of those two, I doubt I can do much about the validation. But I may be able to save some in the email plugin. First I need to profile it to see where the time is going, although I'm pretty sure I know what Ill find ...

-- hugh
 
OK, I may have just found something which will speed up submissions considerably. I've cut your form's time in about half, down from 16+ seconds to about 8 seconds.

Try again, and let me know what you think, and also let e know if you notice any issues with the emailed data.

NOTE to self and rob - I changed form-model.php's getEmailData() to cache in $model->emailData, instead of $this->emailData ... because $this is the specific plugin element object ... so each plugin was rebuilding the entire emailData structure again, which is an expensive operation.

To preserve backward compat for now with anything expecting to find it in the plugin object, I'm copying it, so both plugin and form model objects have a copy. If this fix sticks (and doesn't have unexpected nasty side effects) I'll prolly go back and change everything that references the specific plugin copy, to use the form model copy instead, so we only have it in one place.

-- hugh
 
NOTE - I haven't committed that change yet, so it's only on your server and my server, until I've had time to test it a little, or you find a bug with it.

-- hugh
 
Thanks for the work, it seems to make a difference. Waiting for the validation 'feels' like the biggest thing and I still need to add more in there.
If the client is done with his checks, I want to try if it makes a difference loosing most ajax calcs and do them on form save, see if that helps.
But glad you found at least one reason :)

Can you explain what you did so I can do it on the live site: http://offerte.zuidwijkcarrosserieen.nl/ or if it is not too much to ask can you repeat it there? The difference being PHP 5.4 and more use = better testing on a slower server.
 
The main file I changed is ./components/com_fabrik/models/plugin-form.php, which is where I made that change I talked about in the getEmailData() caching. As long as you are running (more or less) up to date github's on both sites, you should just be able to copy that file over.

I'll wait a few more days to see if you or I hit any unexpected issues due to this change, before I commit it to github. I'm not expecting anything major, but I have a sneaky feeling there may be some issues, to do with calc's, as I have a feeling we may wind up caching the email data (i.e. the submitted data formatted in a way suitable for inserting in emails, which is often different to list, form or details views) before some or all of the calc's have been run, depending what other element types are on the form.

Turning off AJAX wont have any effect on the submission time. It may speed up live use of the form a little, but we're going to run the calcs on submission regardless of AJAX settings.

If you look at the profling info (see previous post on how to do this, basically turn on J! debug, and add &fabrikdebug=2 to the URL that initially loads the form, then look in the Profile section of J!"s debug info at the bottom of the page), you'll get some idea of why the submission process takes some time, when you have so many elements, and in particular, so many calc elements. The amount of processing we have to do is ... well, significant!

Bottom line, you are pushing the envelope waaaaaaaaaaay beyond what we designed it for. We never even considered anyone having this many calc's on a form. So really, I'm actually surprised it's submitting as fast as it is.

I may be able to shave another second or two off that. I noticed that the redirect plugin (which has to run twice, don't ask) is taking 1.5 seconds, which I think is where it stores the form status in the session. I think we may be able to avoid doing that on the second time round, or postpone doing it on the first time round. This has never been an issue before, as it typically only takes a fraction of a second ... but with all those calcs ... yada yada.

But I don't see us being able to get much beyond that, so a reasonable expectation is going to be around 6 seconds, if I can improve the redirect plugin time. I don't think there's going to be much I can do about the validation time, as we've already done a lot of work on making that as efficient as we can, it's just a lengthy process.

-- hugh
 
Hugh thanks a lot for the great explanation. Had some good laugh here too. So in my first project I went over the limits with my 500+ elements on a form and now with the second I go waaay over the max calculations. Too enthusiastic??? Lucky my third project went smooth and within the Fabrik limits :)
Wouldn't know a better way to do this so lets hope I can keep it in the air. Lucky we have fast SSD raided servers.

Would switching of the session store make a difference? (In theory?) I switched it off but did not notice a difference. Switched it back on because I think it is a pretty user friendly system.

My initial goal is to minimise the waiting between the pages. If the end submission takes a bit then that is all int he game I would say but clicking next and waiting 5 secs is annoying with so many pages. But yes, I think you helped to speed it up already a bit.
 
If you tried turning session store off and it didn't make any difference, then the answer to "will it make a difference" is very probably "No". ;)

Yeah, 500 elements on a form was kind of "too enthusiastic". Although with that scenario, the most likely serious problems are going to be MySQL related, hitting hard limits within MySQL for things like maximum query length, etc.

There's no real way for us to provide detailed limits on things, as it depends on a lot of factors beyond our control. And also, in many cases we simply don't know. I've never seen a form with more than a dozen or so calc's on it, and never even thought about the performance issues with having 75+.

Quite honestly, I don't think there's anything I can do to speed up moving between pages, as this involves validation. And even if there's no validations on a given page, we're still going to go through the pre-processing phase, to run all the calcs, and prepare the form for validation / submitting for a variety of gory technical reasons. And as per myu previous post, if yu look at the profiling, between "validation start" and "validation end", you'll get some idea of the amount of processing involved. And that's just the few more obvious contenders for time consumption that I've put profile checkpoints in, there's a whole bunch more going on than shows in the profiling.

About the only way you might speed it up would be to do it as a series of "daisy chained" forms, using copies of the main list/form to emulate the "pages" you have now, where you unpublish everything except the elements on the current "page". But that's a fair amount of work to get going, although simple enough to do a basic proof of concept test - just create a couple of copies for the first two pages, and set up a redirect from "page 1 test" to "page 2 test". The redirect URL from 1 to 2 wuld have to append &rowid={rowid} to the oage 2 URL, so you then edit the partially completed form.

-- hugh
 
Guess the split-up in forms might be the best option. Guess it will still be possible to keep the same calculation calls to the element names. I will make a test.
 
Well, remember that when you unpublish something, it doesn't exist as far as Fabrik is concerned. So if calcs on (say) Page 5 rely on elements from Page 2, then you won't be able to unpublish the ones from Page 2. You'd have to keep them published, but hidden (either by hiding the elements, or hiding the group they are in.

-- hugh
 
We are in need of some funding.
More details.

Thank you.

Members online

No members online now.
Back
Top