Error decoding JSON data: Control character error, possibly incorrectly encoded

Discussion in 'Community' started by xtrgeo, Aug 7, 2017.

  1. xtrgeo

    xtrgeo Member

    Level: Community
    This happens when trying to access a list.

    Already the latest update from github...

    Any ideas?
     

    Attached Files:

  2. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Professional
    OK, that means there's some broken JSON in the params of one of your elements.

    Unfortunately, the only way to get some clue as to which one is to modify some Joomla core code.

    Edit the file:

    ./libraries/vendor/joomla/registry/src/Format/Json.php

    ... and around line 72 will be ...

    Code (Text):

            if ($decoded === null)
            {
                throw new \RuntimeException(sprintf('Error decoding JSON data: %s', json_last_error_msg()));
            }
     
    ... modify that to add the actual data to the error message:

    Code (Text):

            if ($decoded === null)
            {
                throw new \RuntimeException(sprintf('Error decoding JSON data: %s : the bad data is: %s', json_last_error_msg(), $data));
            }
     
    That should give you some clue as to which one it is. Then of course it has to be fixed, and we have to work out how the bad data got in there.

    So do that, and paste the new error message here.

    -- hugh
     
  3. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Professional
    BTW, this was the result of a change in Joomla code in 3.6.3, which now treats malformed JSON as a fatal error, where as prior to 3.6.3 it would just discard the bad data and return "null". This caused a BIG argument at the end of last year:

    https://github.com/joomla/joomla-cms/issues/12460#issuecomment-256219685

    ... where you can see the main authors of J! getting very angry.

    The problem is, although I agree that J! should no longer just silently discard problems in JSON configuration data, I think it should attempt to give some information about what that damaged data is (as per the code change I gave in the previous post) to make it possible to find and fix the problem.

    -- hugh
     
  4. xtrgeo

    xtrgeo Member

    Level: Community
    How! Huge error report...

    The error is too big to fit in the message, so I attach it's rest body in the text file...


    Code (Text):
    Error decoding JSON data: Control character error, possibly incorrectly encoded : the bad data is: {"placeholder":"","password":"0","maxlength":"255","disable":"0","readonly":"0","autocomplete":"1",
    "speech":"0","advanced_behavior":"0","bootstrap_class":"input-medium",
    "text_format":"text","integer_length":"6","decimal_length":"2","field_use_number_format":"0",
    "field_thousand_sep":",","field_decimal_sep":".","text_format_string":"","field_format_string_blank":"1","text_input_mask":"","text_input_mask_autoclear":"0","text_input_mask_definitions":"","render_as_qrcode":"0","guess_linktype":"0","link_target_options":"default","rel":"","link_title":"","show_in_rss_feed":"0","show_label_in_rss_feed":"0","use_as_rss_enclosure":"0","rollover":"","tipseval":"0","tiplocation":"top-left","labelindetails":"0","labelinlist":"0","comment":"","edit_access":"1","edit_access_user":"","view_access":"1","view_access_user":"","list_view_access":"1","encrypt":"0","store_in_db":"1","can_order":"1","alt_list_heading":"","custom_link":"","custom_link_target":"","custom_link_indetails":"1","use_as_row_class":"0","include_in_list_query":"1","always_render":"0","icon_hovertext":"1","icon_file":"","icon_subdir":"","filter_length":"20","filter_access":"1","full_words_only":"0","filter_required":"0","filter_build_method":"0","filter_groupby":"text","inc_in_adv_search":"1","filter_class":"input-medium","filter_responsive_class":"","tablecss_header_class":"","tablecss_header":"","tablecss_cell_class":"","tablecss_cell":"","sum_on":"1","sum_label":"Sum","sum_access":"1","sum_split":"","avg_on":"0","avg_label":"Average","avg_access":"1","avg_round":"0","avg_split":"","median_on":"0","median_label":"Median","median_access":"1","median_split":"","count_on":"1","count_label":"Count","count_condition":"","count_access":"1","count_split":"","custom_calc_on":"0","custom_calc_label":"Custom","custom_calc_query":"","custom_calc_access":"1","custom_calc_split":"","custom_calc_php":"","validations":[],"join_val_column_concat":null,"sum_value_serialized":"a:320:{i:0;O:8:\"stdClass\":4:{s:5:\"value\";s:1:\"0\";s:5:\"label\";s:1:\"0\";s:7:\"elLabel\";s:6:\"risk 0\";s:8:\"calLabel\";s:3:\"Sum\";}s:5:\"Total\";O:8:\"stdClass\":6:{s:5:\"value\";d:914803.55950701237;s:5:\"label\";s:5:\"Total\";s:5:\"class\";s:10:\"splittotal\";s:7:\"special\";b:1;s:7:\"elLabel\";s:10:\"risk Total\";s:8:\"calLabel\";s:3:\"Sum\";}i:1;O:8:\"stdClass\":4:{s:5:\"value\";s:18:\"115.44000160694122\";s:5:\"label\";s:1:\"1\";s:7:\"elLabel\";s:6:\"risk 1\";s:8:\"calLabel\";s:3:\"Sum\";}i:2;O:8:\"stdClass\":4:
     

    Attached Files:

    Last edited: Aug 8, 2017
  5. troester

    troester Well-Known Member Staff Member

    Level: Standard
    Hi Hugh
    I assume it's the "list calculation is stored as session data in the element params..." following our Skype session
    [26.01.2017 23:46:10] B. Tröster: It's replicatable: a list with calculation is modifying the element's params on list load
    (without error now)
    [27.01.2017 00:01:51] Hugh Messenger: yes - when the list loads, it runs the element calculations to be cached, if that list load itself isn't cached.
    [27.01.2017 00:02:18] Hugh Messenger: I can't figure out what was wrong with that string, but it fails an unserialize as well.
    [27.01.2017 00:03:02] Hugh Messenger: theoretically a JSON string can be anything, as long as double quotes are escaped.
     
    Last edited: Aug 8, 2017
  6. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Professional
    @troester - yup, but what's puzzling me is why it's getting this specific error. The issue with the params longer than 65535 should be fixed, although the error msg he attached on the last post shows that the params are truncated. Although I suppose if he just updated, and they haven't been saved since the update. But the error is talking about "control character error".

    @xtrgeo

    OK ...

    So the problem element is the one that has a list calculation with the label "risk Total".

    What you'll have to do is ...

    With something like phpMyAdmin, look at your #__fabrik_elements table, and see what field type the 'params' field is. It should be MEDIUMTEXT, but it may still be TEXT. If it is TEXT, then you need to change the field type to MEDIUMTEXT. This should be safe, you won't lose any data, as it's just changing it to a text field type which can store more data. But if you do need to change it to MEDIUMTEXT, back up your database (or at least this table) first.

    Then find the element in #__fabrik_elements which is the one that has the "risk Total" calculation on it. Edit that row, and look at the params field. If it really does end in:

    Code (Text):

    <span class=\"calclabel\">301
     
    ... rather than something like ...

    Code (Text):

    <span class=\"calclabel\">Total:<\/span> $7,316.34<\/dd>\n<\/dl>"}
     
    ... ie. is closed with a ...

    "}

    ... then you need to delete that calculation entirely, so your params has ...

    "join_val_column_concat":null,"sum_value_serialized":""}

    ... rather than ...

    "join_val_column_concat":null,"sum_value_serialized":"... incredible long string that doesn't have a close quote and curly brace on the end

    Once you've done that, so the JSON is properly formatted again, you should be able to load the list, or edit and save the element settings, and it should fix itself.

    If you params DOES already end with "}, then we need to figure out what is causing the problem.

    -- hugh
     
  7. xtrgeo

    xtrgeo Member

    Level: Community
    Thnks for the detailed reply Hugh.

    I followed your directions and I also had to do the same for another element that had calculations also.

    Thnkfully, there were no "}" at the end of params and I erase the calculation. Save those elements settings again and everything works fine.

    One thing I noticed is that when saving again the element is that it asked it me for confitmation for type change.

    The first from float to varchar(255). I choosed no, changed the type in elements settings as decimal and choosed to be seen as number (Yes/No option).

    Both elemetns changed to decimal 12,2 type.

    Why this happened? Is there any change that something need to be changed in fabrik's code in order for this error message to be seen again?
     
  8. cheesegrits

    cheesegrits Support Gopher Staff Member

    Level: Professional
    It depends what version you updated from. We always check to make sure the data type of the underlying field matches whatever type the element is set to. That code has changed slightly over the years.

    But I don't think we've ever used any of the FLOAT field types. If you choose "decimal" we use DECIMAL.

    So I suspect your fields were set to float by something / someone else.

    And no, there's nothing you need to change to avoid the JSON error.

    The problem was that our original #__fabrik_elements table set the 'params' field to be of type TEXT, which is limited to 65535 characters. It didn't occur to us way back then (10 years ago) that if you have a split calculation, on a large table, with a lot of splits, that when we cache the calculation in the params, it could exceed 64k.

    Prior to Fabrik 3.7, those params would just get truncated, meaning that the JSON was invalid. Prior to J! 3.6.3, when J! reads the params, if the JSON was invalid it just silently discarded it. As of 3.6.3, they throw that fatal exception you saw.

    So ... in Fabrik 3.7, as part of our updater code, we change the params field to MEDIUMTEXT, which allows for a few megabytes. We also double check in our code that writes out those params, and if the length is > 65535, we change the field type. That's in case someone has updated from github, rather than running the actual updater, and the field type hasn't been changed yet.

    Unfortunately, if your params were already truncated when you update to Fabrik 3.7, it's still going to blow up when anything tries to open that element, and there's really no way for us to work round that. But now you've fixed the affected elements by hand, and changed your params to MEDIUMTEXT, it won't happen again.

    -- hugh
     
    Last edited: Aug 9, 2017

Share This Page