Error in scheduler.php

[rbuelund]

Are you going to fix this error ??:

8192,preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead,/home/musiki/public_html/components/com_fabrik/models/importcsv.php,1386

 

[Bren]

I wonder if that's why my Fabrik scheduler stopped working correctly ever since I updated to the current Joomla 3.4.5 & Fabrik 3.3.4?
Thanks in advance.

 

[cheesegrits]

Are you going to fix this error ??:

It's not an "error", it's a deprecated notice which you will only see when running PHP reporting in 'developer' mode. It just means that at some point in the future the /e switch will stop working on preg_replace. Deprecated warnings are simply to give developers notice, so they have plenty of time to replace deprecated syntax. So yes, at some point we will replace that with a pre_replace_callback.

You shouldn't be running that level of error reporting unless you are a developer, as it can cause problems in things like AJAX calls, where deprecated notices will get injected into JSON responses, breaking the Javascript which processes the AJAX response.

On a production site you should turn error reporting off, as it can give hackers information you don't want them having about your site (full paths to code, database details, etc). On a test site, reporting errors and warnings is usually sufficient.

-- hugh

 

[cheesegrits]

I don't really have anything I can test this one, so if someone wants to try this code ...

    protected function charset_decode_utf_8($string)
    {
        /* Only do the slow convert if there are 8-bit characters */
        /* avoid using 0xA0 (\240) in ereg ranges. RH73 does not like that */
        if (!preg_match("/[\200-\237]/", $string) and !preg_match("/[\241-\377]/", $string))
        {
            return $string;
        }

        // Decode three byte unicode characters
        $pattern = "/([\340-\357])([\200-\277])([\200-\277])/";
        $string  = preg_replace_callback(
            $pattern,
            function($m) {
                return '&#' . ((ord($m[1])-224)*4096 + (ord($m[2])-128)*64 + (ord($m[3])-128));
            },
            $string
        );

        // Decode two byte unicode characters
        $string = preg_replace_callback(
            "/([\300-\337])([\200-\277])/",
            function ($m) {
                return '&#' . ((ord($m[1])-192)*64+(ord($m[2])-128));
            },
            $string
        );

        return $string;
    }

... starting around line 1372, and let me know ...

-- hugh

 

[rbuelund]

I don't really have anything I can test this one, so if someone wants to try this code ...

    protected function charset_decode_utf_8($string)
    {
        /* Only do the slow convert if there are 8-bit characters */
        /* avoid using 0xA0 (\240) in ereg ranges. RH73 does not like that */
        if (!preg_match("/[\200-\237]/", $string) and !preg_match("/[\241-\377]/", $string))
        {
            return $string;
        }

        // Decode three byte unicode characters
        $pattern = "/([\340-\357])([\200-\277])([\200-\277])";
        $string  = preg_replace_callback(
            $pattern,
            function($m) {
                return '&#' . ((ord($m[1])-224)*4096 + (ord($m[2])-128)*64 + (ord($m[3])-128));
            },
            $string
        );

        // Decode two byte unicode characters
        $string = preg_replace_callback(
            "/([\300-\337])([\200-\277])",
            function ($m) {
                return '&#' . ((ord($m[1])-192)*64+(ord($m[2])-128));
            },
            $string
        );

        return $string;
    }

... starting around line 1372, and let me know ...

-- hugh

 

[rbuelund]

I saw this in the log file for the scheduler.php - it is not because of my error rapporting. The reason for my question is that all of a sudden the scheduler imports my records severeal times (I import once a day) although I have set it to delete the prior post in the db - there is something wrong with the latest importcsv.php on github !!!

 

[cheesegrits]

Did you try the fix for that notice I suggested?

I doubt it has anything to do with how often your scheduler is running the task, but it should get rid of that notice.

-- hugh

 

[rbuelund]

Tried your code - now i get this warning in the #_fabrik_log table : 2,preg_replace_callback(): No ending delimiter '/' found,/home/musiki/public_html/components/com_fabrik/models/importcsv.php,1398

and line 1398 is the last line in this part:

// Decode two byte unicode characters
$string = preg_replace_callback(
"/([\300-\337])([\200-\277])",
function ($m) {
return '&#' . ((ord($m[1])-192)*64+(ord($m[2])-128));
},
$string
);

 

[cheesegrits]

Ooops. I've corrected the code in that post.

-- hugh

 

[rbuelund]

Ok - but this did not solve my problem. The old rows in the database are not being deleted ! - There must be some kind of bug in the importcsv.php file ??

The odd thing is, that if I trigger the importcsv directly in backend - it works! But not if it is auto triggered by a pageload.

 

[felixkat]

I can confirm there is a bug with this but I haven't looked at it yet.

I started to debug but opened a can of worms as it acts different under windows using Wamp. I'm in the process of building a linux server, (again), so I can debug further. I think the windows issue is to do with the JPATH_ROOT reference but I haven't identified if that's due to my server being setup incorrectly or the importcsv plugin not catering for a windows setup.

On the Linux side I can confirm what the others are seeing. I have a 20 record list which is supposed to refresh hourly and clear existing data. It currently has nearly 5000 entries.... LOL.


I've logged this as it's confirmed and will try and look this week unless somebody jumps in before me.

https://github.com/Fabrik/fabrik/issues/1600

 

[felixkat]

Ok - but this did not solve my problem. The old rows in the database are not being deleted ! - There must be some kind of bug in the importcsv.php file ??

The odd thing is, that if I trigger the importcsv directly in backend - it works! But not if it is auto triggered by a pageload.

Following on from my last post I took a quick look at this. This seems to be a permissions issue, running it manually works because it's likely you are an admin the backend.

Running from a cron fails unless the list has 'public' access for 'Empty records'.

Note to Hugh:-

The importcsv model checks if $dropData is set and if you have permission to empty tables.

        if ($dropData && $model->canEmpty())
        {
            $model->truncate();
        }

The list model checks the permission for canEmpty().

    public function canEmpty()
    {
        if (!array_key_exists('allow_drop', $this->access))
        {
            $groups = $this->user->getAuthorisedViewLevels();
            $this->access->allow_drop = in_array($this->getParams()->get('allow_drop'), $groups);
        }

        return $this->access->allow_drop;
    }

...which will fail via cron because cron is effectively 'public' so will only delete if the list has 'public' set for deleting the list.

Maybe we could use 'Query Secret' as an override?

 

[cheesegrits]

Thought I'd replied to this yesterday. Yup, it's probably a permissions issue, which is something which has always been problematic with cron jobs, along with prefilters.

I'm looking at a couple of potential workarounds.

Hugh




Sent from my Nexus 7 using Tapatalk

 

[rbuelund]

Anything new on this ?

And a little update. I have two almost identical sites running on the same server with PHP 5.5.
Both installations run Joomla 3.4.8. The only difference is that the site with problems has had fabrik updated lately from github the other runs the latest stable version 3.3.4 ??
I can see that there is no difference in the importcsv.php file on the two sites - so the error is not in there. But something must have been changed somewhere else in fabrik, since this problem arises out of the blue ???

 

[felixkat]

I've been playing with the code in terms of allowing us to 'truncate' via a wget request. It's the truncate that is failing at the moment, i.e an import appends to an existing list instead of replacing it. To make the 'wget' a little more secure it will only work with querystring and secret enabled, this effectively gives the code the permissions to perform the truncate request.

I haven't looked at a regular cron job yet, i.e one that should be triggered internally, (when somebody uses the site).

I think you are right when you say it used to work as I had something similiar setup but I haven't looked at what stopped it working yet, although I will try tonight as I need this as well.

It's not only the importcsv.php code that is involved here as it links in with the cron plugin and list models.

 

[troester]

There was a fix to prevent regular CSV import dropping data if the user don't have "empty" privilege.
Maybe this is involved here?
https://github.com/Fabrik/fabrik/commit/f5cee3ded26b4d1c64db624e853bdad7f107576a

 

[cheesegrits]

Rbueland - can you confirm whether the 'empty table' and delete rows access are the same on both servers? As per this discussion, remember that cron jobs run with the access of whoever is logged in when the job runs.


Sent from my HTC One using Tapatalk

 

[rbuelund]

Rbueland - can you confirm whether the 'empty table' and delete rows access are the same on both servers? As per this discussion, remember that cron jobs run with the access of whoever is logged in when the job runs.


Sent from my HTC One using Tapatalk

Well it is two different webhotels on the same shared server - so the access must be the same. The one site is a copy of the other site.

 

[troester]

I think @cheesegrits means the list access settings in each of your lists involved.

 

[rbuelund]

They are exactly the same on both sites