cheesegrits
Support Gopher
Yes, it's because you didn't update when i said to update.
I'll need phpMyAdmin access.
Make sure you have those details in My Sites, and let me know which site.
-- hugh
-
New Commercial Services Section
We have now opened a commercial services section here on the forum for registered users. If you have a Fabrik project that you wish to have someone work on for you, post it under Help Wanted. If you are an application developer and wish to earn some money helping others, post your details under Fabrik Application Developers.
Both of these are unmoderated. It will be up to both parties to work out the details and come to an agreement.
-
Joomla 5.1
For running J!5.1 you must https://fabrikar.com/forums/index.php?wiki/update-from-github/ or include the new file manually https://fabrikar.com/forums/index.php?threads/joomla-5-1-and-fabrik-cannot-find-files-error.54473/post-285151 See also Announcements
Got Error decoding JSON data: Syntax error while accessing elements
- Thread starter 7webcreator
- Start date
Yes, it's because you didn't update when i said to update.
I'll need phpMyAdmin access.
Make sure you have those details in My Sites, and let me know which site.
-- hugh
7webcreator
Member
yes, I updated the site after, sorry, sorry. The credentials of the accademiadiformazionemusicale.com cpanel are now in My Sites. Just to let you know, in the meantime, I have replaced the params of id 1692 with the params of one of my previous backups and the list works again. However, I would not like to get this issue again. Could you please see what is going wrong? Thank you for your time.
cheesegrits
Support Gopher
OK, well as long as you have updated from github, you should now see an error (runtime exception) if the element is saved with a string that is too long, saying "Element params too big to save, probably calculation data".
When that happens, we can take a look at the configuration of that element and the list it is in, and figure out why that calculation is generating such a huge serialized value. I suspect it may be to do with grouping on the list and/or the "split", and may not be entirely trivial to solve, but we'll figure it out.
One solution, which may well be what we have to do, would be to change the type of the 'params' field in #__fabrik_elements from TEXT to MEDIUMTEXT, which would raise the maximum length from 64K bto 16Mb.
-- hugh
When that happens, we can take a look at the configuration of that element and the list it is in, and figure out why that calculation is generating such a huge serialized value. I suspect it may be to do with grouping on the list and/or the "split", and may not be entirely trivial to solve, but we'll figure it out.
One solution, which may well be what we have to do, would be to change the type of the 'params' field in #__fabrik_elements from TEXT to MEDIUMTEXT, which would raise the maximum length from 64K bto 16Mb.
-- hugh
cheesegrits
Support Gopher
Note that I just made a small change to that error handling, to add the element ID to the error.
https://github.com/Fabrik/fabrik/commit/189363be865271bbcfd30fd0a2892ddf7721a419
-- hugh
https://github.com/Fabrik/fabrik/commit/189363be865271bbcfd30fd0a2892ddf7721a419
-- hugh
7webcreator
Member
Thank you for your reply.
I should let you know that looking at those 3 elements, I can assure that there are no splits or filters. Only the list is grouped with one element that is not involved in the issue I encounter. However, I noticed sometimes at the bottom of the page list a huge display of IDs and sums.
I should let you know that looking at those 3 elements, I can assure that there are no splits or filters. Only the list is grouped with one element that is not involved in the issue I encounter. However, I noticed sometimes at the bottom of the page list a huge display of IDs and sums.
Last edited:
cheesegrits
Support Gopher
OK, I had a look at your site.
I think you may simply be hitting a limit we didn't anticipate when we started. If your list is grouped, then the calculations are grouped, so we can show the values for each group in your list. And rather than re-calculate everything on the fly every time you display the list, we only calculate when anything is changed (like submitting a form, saving changes to lists or elements on the back end, etc), and we store the values in that "serialized" attribute in the params field for the element.
So of course, the size of that serialized value grows, as your table grows, and the number of groupings increases. If you have a small list, grouped on (say) "name", and you only have three different names ... we only have to store 3 calculation values. But by the time you have tens of thousands of rows with hundreds or thousands of different names .... we have to store hundreds or thousands of grouped calculation values.
I think the only solution is to change the type of the 'params' field in our fabrik_elements table from TEXT to MEDIUMTEXT. This won't increase the size of the table, it just allows that field to grow much bigger.
So ... what I suggest is that you do a full database backup, then in phpMyAdmin we change the type of your params field. And I'll whip up the SQL update script we'll need in our next release build to change that type for all users, when they next update Fabrik.
Re:
That should only happen if you have a "split" on your calculation, as splits are another case in which we have to store multiple calculations in the serialized data.
-- hugh
I think you may simply be hitting a limit we didn't anticipate when we started. If your list is grouped, then the calculations are grouped, so we can show the values for each group in your list. And rather than re-calculate everything on the fly every time you display the list, we only calculate when anything is changed (like submitting a form, saving changes to lists or elements on the back end, etc), and we store the values in that "serialized" attribute in the params field for the element.
So of course, the size of that serialized value grows, as your table grows, and the number of groupings increases. If you have a small list, grouped on (say) "name", and you only have three different names ... we only have to store 3 calculation values. But by the time you have tens of thousands of rows with hundreds or thousands of different names .... we have to store hundreds or thousands of grouped calculation values.
I think the only solution is to change the type of the 'params' field in our fabrik_elements table from TEXT to MEDIUMTEXT. This won't increase the size of the table, it just allows that field to grow much bigger.
So ... what I suggest is that you do a full database backup, then in phpMyAdmin we change the type of your params field. And I'll whip up the SQL update script we'll need in our next release build to change that type for all users, when they next update Fabrik.
Re:
That should only happen if you have a "split" on your calculation, as splits are another case in which we have to store multiple calculations in the serialized data.
-- hugh
7webcreator
Member
I understand. As you could see attached there is no split on the calc element id 1683. This id belongs to a list that is grouped by teacher and I'm happy with the grouping of calculations by teacher when the page does not display the serialised data.
I have now back-up the database as requested.
I have now back-up the database as requested.
cheesegrits
Support Gopher
How many teachers do you have?
Now you've backed up, use phpMyAdmin to change the field type of the 'params' field in the #__fabrik_elements table from TEXT to MEDIUMTEXT.
-- hugh
Now you've backed up, use phpMyAdmin to change the field type of the 'params' field in the #__fabrik_elements table from TEXT to MEDIUMTEXT.
-- hugh
7webcreator
Member
There are 24 teachers and the list contains all the sessions they are delivering for the all year.
I have now changed the params as specified above. I did not encounter any further issues. Should I have to update too using this https://github.com/Fabrik/fabrik/commit/189363be865271bbcfd30fd0a2892ddf7721a419 ?
I have now changed the params as specified above. I did not encounter any further issues. Should I have to update too using this https://github.com/Fabrik/fabrik/commit/189363be865271bbcfd30fd0a2892ddf7721a419 ?
cheesegrits
Support Gopher
Hmm, 24 shouldn't break anything.
Anyway ... if you've set that params field to MEDIUMTEXT, you shouldn't get this problems again. That raises the maximum length for the params field from 64Kb to 16Mb. If we ever hit that limit, we'll have bigger problems than this thread.
So no, you don't really need to update, as all that change does is adds the affected element ID to the error message about the params being too long, which you shouldn't ever see now, with the larger field.
So just wait till you next need to update to fix something else.
-- hugh
Anyway ... if you've set that params field to MEDIUMTEXT, you shouldn't get this problems again. That raises the maximum length for the params field from 64Kb to 16Mb. If we ever hit that limit, we'll have bigger problems than this thread.
So no, you don't really need to update, as all that change does is adds the affected element ID to the error message about the params being too long, which you shouldn't ever see now, with the larger field.
So just wait till you next need to update to fix something else.
-- hugh
7webcreator
Member
Thank very much for your straight forward guidance. I hope this issue won't come back again
dimoss
Well-Known Member
Hi,
I re-open this old thread because i had the same problem with JSON error.
I fixed following Hugh's suggestion and then I updated from Github to F3.6 on my J3.6.5 site.
What I realized is that the problem still exists when there is a "split" on any calculation in an element.
In my case there were two elements, one field and one dbjoin, with "splits" on Count calculation. Both stored multiple calculations in the serialized data and that seems made these two element to grow and exceed the 65K limit of TEXT type.
What I did was to remove the "split" and change to MEDIUMTEXT.
However I think that it should be safer if we could have a more permanent solution on this problem.
I re-open this old thread because i had the same problem with JSON error.
I fixed following Hugh's suggestion and then I updated from Github to F3.6 on my J3.6.5 site.
What I realized is that the problem still exists when there is a "split" on any calculation in an element.
In my case there were two elements, one field and one dbjoin, with "splits" on Count calculation. Both stored multiple calculations in the serialized data and that seems made these two element to grow and exceed the 65K limit of TEXT type.
What I did was to remove the "split" and change to MEDIUMTEXT.
However I think that it should be safer if we could have a more permanent solution on this problem.
cheesegrits
Support Gopher
As I said on Skype last night, there was a fix for this about six months ago. It's in two parts. For now, there's a check where we save those params which will throw an exception rather than try and write out a params longer than the field:
https://github.com/Fabrik/fabrik/commit/d07b2138c83d1012f953534fa6fea5a859dd5c28
... and in the Fabrik 3.6 update SQL, we increase the size of the params field to MEDIUMTEXT.
At some point soon I'll add a test on the length check so we don't do that if you are running >= J! 3.7.
-- hugh
https://github.com/Fabrik/fabrik/commit/d07b2138c83d1012f953534fa6fea5a859dd5c28
... and in the Fabrik 3.6 update SQL, we increase the size of the params field to MEDIUMTEXT.
At some point soon I'll add a test on the length check so we don't do that if you are running >= J! 3.7.
-- hugh
dimoss
Well-Known Member
Hi Hugh,
I updated to F3.6 via Github so I suppose I didn't run any SQL update. I changed the params data type manually on _elements, _forms and _list.
Am I need to do something more?
If I activate the split again it will store the huge serialized data again. These lists have big traffic. Does it means that every query will add data on the params field? If yes then sooner or later i will hit the max again.
Thanks!
I updated to F3.6 via Github so I suppose I didn't run any SQL update. I changed the params data type manually on _elements, _forms and _list.
Am I need to do something more?
If I activate the split again it will store the huge serialized data again. These lists have big traffic. Does it means that every query will add data on the params field? If yes then sooner or later i will hit the max again.
Thanks!
cheesegrits
Support Gopher
No, every query doesn't add to it, it replaces it.
Sent from my HTC6545LVW using Tapatalk
Sent from my HTC6545LVW using Tapatalk
dimoss
Well-Known Member
Thanks for the reply. This sounds good!