Slow loading for Googlemap Visualization

A

abel408

Member
I've been experiencing very slow loading times when loading my google map visualization of around 400 records. I think it is getting stuck on some sort of query. I've done some troubleshooting, but can't figure much out. Hoping someone here know more than I do when troubleshooting this kind of stuff.

Here is my output from the debug tools of Google Chrome:
rJR6Odr.png


As you can see, it takes 9.5 seconds before the browser sees the first byte of data. Wayyy too long.


When viewing the log of mysql, it seems to get stuck during this process:

FROM `hibre_places`

90 QuerySELECT FOUND_ROWS()

*Extremely Long Wait Time*

160303 15:23:29 88 QuerySELECT id, label, db_table_name

FROM hibre_fabrik_lists

88 QuerySELECT l.db_table_name,

el.name, el.plugin, l.label AS listlabel, l.id as list_id,


el.id AS element_id, el.label AS element_label, f.id AS form_id,

el.params AS element_params​


This only seems to happen during the loading of the Google map visualization. The list view has no issues with time.

Last, but not least, here is my Joomla Debug:

Time: 0.0 ms / 0.0 ms Memory: 0.383 MB / 0.38 MB Application: afterLoad
Time: 22.8 ms / 22.8 ms Memory: 2.105 MB / 2.49 MB Application: afterInitialise
Time: 87.0 ms / 109.8 ms Memory: 0.239 MB / 2.73 MB Application: afterRoute
Time: 27.2 ms / 137.0 ms Memory: 1.119 MB / 3.85 MB Application: start get Request data
.
.
.
Time: 0.6 ms / 203.3 ms Memory: 0.006 MB / 9.48 MB Application: runPlugins: start: onCanEdit
Time: 0.0 ms / 203.3 ms Memory: 0.003 MB / 9.48 MB Application: runPlugins: end: onCanEdit


The last 2 lines repeat about 1000 times. Then I get this:

Time: 0.6 ms / 2914.9 ms Memory: 0.000 MB / 18.00 MB Application: elements renderListData: (380) tableid = 1 hibre_places___TypeOfRelationship
Time: 238.3 ms / 3153.2 ms Memory: 0.035 MB / 18.04 MB Application: elements renderListData: (380) tableid = 1 hibre_places___map_marker
Time: 355.8 ms / 3509.0 ms Memory: 0.044 MB / 18.08 MB Application: elements renderListData: (380) tableid = 1 hibre_places___Active_Business
Time: 171.4 ms / 3680.4 ms Memory: 0.004 MB / 18.08 MB Application: elements renderListData: (380) tableid = 1 hibre_places___Job_Types_Offered
Time: 291.3 ms / 3971.7 ms Memory: 0.035 MB / 18.12 MB Application: elements renderListData: (380) tableid = 1 hibre_places___Last_Contact
Time: 213.4 ms / 4185.1 ms Memory: 0.029 MB / 18.15 MB Application: elements renderListData: (380) tableid = 1 hibre_places___Address
Time: 351.0 ms / 4536.0 ms Memory: 0.008 MB / 18.15 MB Application: elements renderListData: (380) tableid = 1 hibre_places___Location

Then "onCanEdit" repeats about another 1000 times. Then this:

Time: 0.6 ms / 5139.9 ms Memory: 0.000 MB / 21.18 MB Application: elements renderListData: (380) tableid = 1 hibre_places___Primary_Number_Email
Time: 371.5 ms / 5511.4 ms Memory: 0.003 MB / 21.18 MB Application: elements renderListData: (380) tableid = 1 hibre_places___Primary_Contact
Time: 345.1 ms / 5856.5 ms Memory: 0.003 MB / 21.19 MB Application: elements renderListData: (380) tableid = 1 hibre_places___Staff_Contact
Time: 335.4 ms / 6191.9 ms Memory: 0.004 MB / 21.19 MB Application: elements renderListData: (380) tableid = 1 hibre_places___individuals_connected
Time: 355.1 ms / 6547.0 ms Memory: 0.002 MB / 21.19 MB Application: elements renderListData: (380) tableid = 1 hibre_places___Additional_Information
Time: 360.2 ms / 6907.2 ms Memory: 0.007 MB / 21.20 MB Application: elements rendered for table data
Time: 0.4 ms / 6907.6 ms Memory: 0.002 MB / 21.20 MB Application: table grouped-by applied



That's as much debugging as I've done. Any help is greatly appreciated! If you need any more information, please let me know.
 
My Memory usage seems pretty high too: 38.86 MB (40,751,192 Bytes)
Compared to the list view: 9.78 MB (10,257,840 Bytes)
 
Have you compared it with rendering the list itself when displaying 400 rows at once (no pagination)?

List rendering is a very compute intensive process, and most of the time is spent rendering each element for display - eg taking the "raw" value from the query, and running it through the rendering process for formatted display in the list.

Your option is basically to ONLY query and render the elements you actually need on the map. Copy the list (select it on the backend and hit Copy), and give the list/form/groups we create some distinct name so you can tell them apart on the backend (it'll ask if you want to rename them). You'll then have a separate set of list/form/groups/elements, sitting on the same table, which you can configure independently. Use that copy for your viz.

Then chop the list back to only the elements you need for the map. Unpublish anything you really don't need at all, or use the "include in list query" and "always render" options on the "list settings" so you only query elements you need, and only render those you are actually going to display (in, say, the pin popup). Sometimes you need to query a value because you use it in (say) a filter, but you don't need it rendered for display.

-- hugh
 
Thanks, creating a new list an unpublishing elements definitely helps. I was able to cut the loading time in half to 4.5 seconds.

Still longer than I would like. I wish the page would load first and then display a little loading icon similar to the ajax loading for list view. At least then users know the system is working and loading the data.

One issue with unpublishing elements. When the user clicks a a record from the googlemap visualization, they are only given the elements that are published, since the form is rendered from this new list. I would like it so all elements are displayed just as if you were to click a record from the original list. Is there any way I can make that happen?

Also, will a faster server help at all? The system it's on now has 8gb of memory and a 4 core 2.4GHz CPU. When I look at usage while loading the map, one of the CPU cores shoots up to 100% while the others stay at zero. If a faster server will help, what would you recommend?

Is a 4.5 second loading time normal for loading 380 records and 7 elements?

Thanks for the help!
 
Depending on what the user can click to view the record: add a custom view link in element settings or in list/Links settings pointing to your original form.

Time:
one part is fetching the data from the database: how long needs the "map" list for loading (if rows per page is set to >400)?
one part is the connection to Google for getting the markers: are you displaying all single markers or using clustering?
If you have a lot of calls you may need a Google Maps API key (you can include it in Fabrik Options)
 
troester said:
Depending on what the user can click to view the record: add a custom view link in element settings or in list/Links settings pointing to your original form.

Time:
one part is fetching the data from the database: how long needs the "map" list for loading (if rows per page is set to >400)?
one part is the connection to Google for getting the markers: are you displaying all single markers or using clustering?
If you have a lot of calls you may need a Google Maps API key (you can include it in Fabrik Options)
Click to expand...


Thanks. The custom links worked great. I ended up just linking to the details view for each user and adding an edit button to the top of the page for users with write access.

I am using clustering. I've played around with the setting and there doesn't seem to be a difference in loading time... or if there is, it's very marginal. I did a test with no clustering and a test with clustering value of 100. They both took about the same amount of time. I have clustering enabled anyway because of visual appeal.

I do not have a Google Maps api. Is there a way to tell if this is a bottleneck?

My theory is that the queries are causing the slow loading times. I'm sure a faster cpu will help things since I'm seeing 100% usage on one core during the query of all the records. Maybe that can some how be divided between cores to speed things up? I don't know much about application and system design to know that answer.

The original list takes just about the same time to load as the google map visualization with the new copied list. It's just a lot more helpful because with the list view, users see a loading icon with the ajaxed list. Do you know of a way to add a loading icon to the Google Map Visualization?

Thanks for all the help thus far!
 
I'm still trying to speed up loading times. I haven't been able to reduce it lower than 5 seconds. According to Joomla Debugging tools, the following are taking the longest to run:

Time: 73.7 ms / 107.1 ms Memory: 0.232 MB / 2.96 MB Application: afterRoute
Time: 239.3 ms / 1799.9 ms Memory: 0.036 MB / 14.43 MB Application: elements renderListData: (385) tableid = 10 hibre_places___map_marker
Time: 374.9 ms / 2174.8 ms Memory: 0.045 MB / 14.47 MB Application: elements renderListData: (385) tableid = 10 hibre_places___Active_Business
Time: 179.9 ms / 2354.7 ms Memory: 0.004 MB / 14.48 MB Application: elements renderListData: (385) tableid = 10 hibre_places___Job_Types_Offered
Time: 301.6 ms / 2656.4 ms Memory: 0.036 MB / 14.51 MB Application: elements renderListData: (385) tableid = 10 hibre_places___Last_WDS_Contact
Time: 206.3 ms / 2863.2 ms Memory: 0.033 MB / 14.56 MB Application: elements renderListData: (385) tableid = 10 hibre_places___Location
Time: 412.4 ms / 5060.0 ms Memory: 0.507 MB / 24.92 MB Application: runPlugins: start: loadJavascriptClass​

And OnCanDelete and OnCanEdit take up a lot of time. Each one is only about .7ms, but it all adds up.

What is interesting is the following "renderListData" takes a fraction of the time compared to the other elements:

Time: 0.1 ms / 176.8 ms Memory: 0.004 MB / 8.38 MB Application: elements renderListData: (385) tableid = 10 hibre_places___id
Time: 0.6 ms / 791.2 ms Memory: 0.000 MB / 11.95 MB Application: elements renderListData: (385) tableid = 10 hibre_places___Name
Time: 0.6 ms / 1560.7 ms Memory: 0.000 MB / 14.39 MB Application: elements renderListData: (385) tableid = 10 hibre_places___TypeOfRelationship​


Why is that? There is absolutely no difference between the element TypeOfRelationship and Job_Types_Offered, yet one takes .6ms and the other takes 179.9ms.

If I run the following query, I am given all records instantly:


SELECT SQL_CALC_FOUND_ROWS DISTINCT `hibre_places`.`id` AS `hibre_places___id`,

`hibre_places`.`id` AS `hibre_places___id_raw`,

`hibre_places`.`Name` AS `hibre_places___Name`,

`hibre_places`.`Name` AS `hibre_places___Name_raw`,

`hibre_places`.`TypeOfRelationship` AS `hibre_places___TypeOfRelationship`,

`hibre_places`.`TypeOfRelationship` AS `hibre_places___TypeOfRelationship_raw`,

`hibre_places`.`map_marker` AS `hibre_places___map_marker`,

`hibre_places`.`map_marker` AS `hibre_places___map_marker_raw`,

`hibre_places`.`Active_Business` AS `hibre_places___Active_Business`,

`hibre_places`.`Active_Business` AS `hibre_places___Active_Business_raw`,

`hibre_places`.`Job_Types_Offered` AS `hibre_places___Job_Types_Offered`,

`hibre_places`.`Job_Types_Offered` AS `hibre_places___Job_Types_Offered_raw`,

`hibre_places`.`Last_WDS_Contact` AS `hibre_places___Last_WDS_Contact`,

`hibre_places`.`Last_WDS_Contact` AS `hibre_places___Last_WDS_Contact_raw`,

`hibre_places`.`Location` AS `hibre_places___Location`,

`hibre_places`.`Location` AS `hibre_places___Location_raw`,

`hibre_places`.`id` AS slug

, `hibre_places`.`id` AS `__pk_val`


FROM `hibre_places`

ORDER BY `hibre_places`.`Last_WDS_Contact` DESC
Click to expand...
This is the same exact query that is run when generating the google map and list. Why does this query come up instantly in mysql, but takes forever in Fabrik?​

Thanks for any help you can provide!
 
As I said, it's the rendering that takes the time.

I doubt very much the actual query takes any more time running from Fabrik than in a mysql client. It's just the processing of the data after loading the query.

The onCanDelete and onCanEdit are a good example. We have to run those at least once for every element being rendered. So 400 rows times (say) 10 elements, that's 8000 calls just to figure out the ACL's.

Remember that Fabrik is not only having to generate a web page, it's an "application builder", so it has to build the entire application from first principles on every page load. That produces compromises and is less efficient than an application coded specifically to do what it does from scratch. At the end of the day, if that trade off doesn't work for you, then your option is to code it from scratch. Not trying to be an ass, that's just the trade off - coding time vs efficiency.

I doubt very much rendering 400 rows into a map you will get it down much further than that. I have a map with about 2,000 rows I use as a test case, which renders in about 3 seconds (in a relatively slow VirtualBox VM) but I'm only rendering 3 simple elements. If I spent some time working on your site I could maybe shave another second or two off the load time, as there are probably some tricks that could be done, but that's not something I can do in Community support, that's Pro support work.

There is absolutely no difference between the element TypeOfRelationship and Job_Types_Offered, yet one takes .6ms and the other takes 179.9ms.
Click to expand...

I'm willing to bet there is a difference, as "identical" things obviously don't consistently take radically different times to run. It just probably isn't an obvious one. For example, are you using one as a filter? if so, are you using "recorded data" or "all data"?

-- hugh
 
Makes sense. If it becomes too much of an issue and I can't get the rendering time to go down, I'll have to look into something else.

To be specific, I have 400 records and the Google Map is loading a total of 8 elements. Like I said before, 3 of these elements load up extremely fast, while the others do not. I'm trying to pinpoint what is different about these elements, but cannot. I've removed all of them as filters and added all of them as filters. Whether they are filters or not, doesn't seem to make a difference. Some of them seem to be identical! I've compared each page of the element side by side and there is nothing different.

In the case of the element" TypeOfRelationship", this element loads extremely fast. As a test, I copied this element and renamed it, did nothing else. The copied version of this element loads slow just like the others. Any way to explain this?

Thanks!
 
Ok... I can't explain this, but I figured out why some elements load faster than others according to Joomla Debug Tools. It all depends on the order. I've changed the order of some elements and now the loading times for each element is different.
 
Hi,

I am also experiencing a very slow loading map for this year 2017. We are displaying an element on google map which is used to represent the attacks happened in current year. Every year we are using the same functionality to build the map and in this year too.

The issue we are facing now is page loads continuously for more than 10mins which is really bad and many people were complains on performance. The 2016 year map contains near about 1300 records, but still it loads in fraction of seconds. Our clients are much worried about this and escalating on us.

To cross check about 2017 year map, I have created 3 times the complete process of map, Still no luck.

Your help is much appreciated. Thanks in advance.
 
Hi, load time for that particular map is very high.

Application 1.001 seconds (+0.015); 23.23 MB (-0.006) - elements rendered for table data
Application 1.004 seconds (+0.003); 23.23 MB (+0.001) - table groupd by applied
Application 1.077 seconds (+0.074); 23.49 MB (+0.256) - end format data
Application 1.077 seconds (+0.000); 23.49 MB (-0.001) - data formatted
Application 421.601 seconds (+420.524); 23.53 MB (+0.036) - fabrik makeFilters start
Application 421.602 seconds (+0.000); 23.53 MB (+0.003) - fabrik makeFilters end
Application 421.654 seconds (+0.053); 24.34 MB (+0.808) - afterDispatch
Application 421.660 seconds (+0.006); 24.66 MB (+0.324) - beforeRenderModule mod_menu (Footer Menu)
Application 421.667 seconds (+0.007); 24.83 MB (+0.173) - afterRenderModule mod_menu (Footer Menu)
Application 421.667 seconds (+0.000); 24.83 MB (-0.007) - beforeRenderModule mod_custom (Cookie Law CCS)
Application 421.669 seconds (+0.001); 24.83 MB (+0.007) - afterRenderModule mod_custom (Cookie Law CCS)
Application 421.669 seconds (+0.000); 24.83 MB (+0.000) - beforeRenderModule mod_menu (Learn More)
Application 421.670 seconds (+0.001); 24.84 MB (+0.006) - afterRenderModule mod_menu (Learn More)
Application 421.670 seconds (+0.000); 24.84 MB (-0.003) - beforeRenderModule mod_peltho_popupdialog (Peltho Popup Dialog)
Application 421.670 seconds (+0.001); 24.86 MB (+0.022) - afterRenderModule mod_peltho_popupdialog (Peltho Popup Dialog)
Application 421.670 seconds (+0.000); 24.85 MB (-0.013) - beforeRenderModule mod_menu (Learn More )
Application 421.680 seconds (+0.009); 24.89 MB (+0.046) - afterRenderModule mod_menu (Learn More )
Application 421.680 seconds (+0.000); 24.89 MB (-0.006) - beforeRenderModule mod_custom (Top Menu)
Application 421.680 seconds (+0.000); 24.89 MB (+0.004) - afterRenderModule mod_custom (Top Menu)
Application 421.680 seconds (+0.000); 24.89 MB (+0.004) - beforeRenderModule mod_search (Quick Search)
Application 421.682 seconds (+0.002); 24.92 MB (+0.023) - afterRenderModule mod_search (Quick Search)
Application 421.699 seconds (+0.017); 25.05 MB (+0.129) - afterRender

Please help me how to resolve the performance issue.
 
I've sort of given up on my issue and I've accepted the limitation of Fabrik. Takes about 6 seconds to load ~500 records on the map. I'm looking to get off of the google map visualization though and use something like AngularJS for the frontend.
 
BTW, did you see the option I added recently for "delay loading" on the map viz?

What that does is instead of loading your 500 rows during page load, and hence delaying the entire page rendering, it doesn't load any rows / markers on page load, and instead fires off an AJAX call once the page and the map are loaded, to load the rows and create the markers, and it puts up a loading spinner on the map until the markers are loaded.

-- hugh
 
sobha said:
Hi,

I am also experiencing a very slow loading map for this year 2017. We are displaying an element on google map which is used to represent the attacks happened in current year. Every year we are using the same functionality to build the map and in this year too.

The issue we are facing now is page loads continuously for more than 10mins which is really bad and many people were complains on performance. The 2016 year map contains near about 1300 records, but still it loads in fraction of seconds. Our clients are much worried about this and escalating on us.

To cross check about 2017 year map, I have created 3 times the complete process of map, Still no luck.

Your help is much appreciated. Thanks in advance.
Click to expand...

I can't do any hands-on help in Community for this. If you'd like to take out a support sub, I can take a look.

-- hugh
 
cheesegrits said:
BTW, did you see the option I added recently for "delay loading" on the map viz?

What that does is instead of loading your 500 rows during page load, and hence delaying the entire page rendering, it doesn't load any rows / markers on page load, and instead fires off an AJAX call once the page and the map are loaded, to load the rows and create the markers, and it puts up a loading spinner on the map until the markers are loaded.

-- hugh
Click to expand...
Thanks, that does help a lot.
 
Hi,

It is having value "Full" for the drop down Use Static Map. Please find the attachment.
 

Attachments

  • googlemap-options.png
    googlemap-options.png
    181.5 KB · Views: 40
@abel408,ck.

i didn't find any delay option in visualization for loading the map. I have changed the map settings as "Static", still no luck.
 
@sobha :
Your screenshot is showing Joomla2.5/Fabrik3.0 which is not longer supported.
Set Static Map = no (so in list view it's showing coordinates, no map)
 
You must log in or register to reply here.

Useful links

  • Create a Fabrik Account (Downloads)
  • Create a Forum Account
  • Fabrik Wiki
  • Fabrik 4 Changelog /Install /Upgrade
    • We are in need of some funding.
    More details.

    Thank you.

    Members online

    Total: 48 (members: 3, guests: 45)
    Back
    Top