Page MenuHomePhabricator

Investigate Civi Load Time issue
Closed, ResolvedPublic2 Estimated Story Points

Description

The Civi homepage takes a really long time to load. Up to 37 seconds on any given morning, and as long as 3 minutes in the mid to late afternoon. (All times Pacific). Can we look into why, and see about a fix? I would categorize this as a repeated annoyance, and a barrier to adoption by new staff/new users.

Event Timeline

Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJun 21 2016, 5:16 PM

I hit a 13 second delay on staging today and the query was per

https://issues.civicrm.org/jira/browse/CRM-19027

I'm pulling this into the sprint since I did a fix for that query, but I think we should do QA against the upstream PR

https://github.com/civicrm/civicrm-core/pull/8636

& once someone approves (ie. comments to say if looks all good for wmf) then I can do a gerrit on our repo (& self-approve that). I think this is the way we should start to do Civi QA going forwards to reduce double up work - the added bonus being maybe a non-WMF person will do the QA for us.

Eileenmcnaughton set the point value for this task to 2.

Email to frtech on this

Hi all,

As you know Caitlin et al have complained about the time it takes to get into CiviCRM the first time each day. ( https://phabricator.wikimedia.org/T138334)

I believe the reason is that CiviCRM is doing it’s ping back.

Basically it reports back to the mother ship various information:

  • unique hash (for anonymisation although really we could be easily picked out by looking at other things)
  • CMS
  • language
  • default country
  • CiviCRM version
  • PHP version
  • Mysql version
  • communityMessagesUrl (this is the dashlet that shows blogs etc and can be changed to point to something else - e.g our own messages, dash, whatever)
  • types of payment processors in use
  • number of contacts, contributions, activities, & some other entities we don’t use much in the DB
  • extensions being used including version

To address the performance side of this we have 2 options

  1. hack the ping backs out
  2. schedule a cron to do them at a more convenient time (it won’t do ping backs during user session if the cron is running & working)

In terms of how the information is used by CiviCRM it basically falls into 2 categories

  1. Marketing - CivCRM regularly publishes figures of the composite number of contacts, contributions etc managed by CIviCRM software, as well as the total number if instances
  2. Decisions
    • can we ditch php 5.3 yet?
    • is this payment processor still being used or can it be removed from core
    • is this extension every used with that language or can we put that problem on the back burner
    • are large sites using this extension

While I have some reservations about the fact there is no longer an opt out from WMF’s point of view I have a couple of notes

  1. They aren’t really gathering any info that we don’t publish in one form or another
  2. I have some doubts that they filter out staging sites - so they might be double reporting us
  3. Actually the cron will enable by default on this screen https://civicrm.wikimedia.org/civicrm/admin/job?reset=1 once we upgrade to 4.7 - so it’s a question of do we want to DISABLE it
  4. This is kind of an addendum to 2 - since we don’t run cron on staging unless we resolve making sure it doesn’t run on staging it might still be slow on staging. There is a ticket that kind of relates https://issues.civicrm.org/jira/browse/CRM-18231
  5. I did identify one unnecessary point of slowness https://github.com/civicrm/civicrm-core/pull/8636

From here

  • If we are happy about the stats nothing needs to be done at this stage, upgrading to 4.7 will improve this on live but not on staging without intervention. I may need to follow up slowness on staging (which does not run the cron)
  • If we are not happy I need to hack it out somehow

Eileen

Some questions how this will work on live with proxy - let's see after the deploy

The deploy definitely did not improve this so I will dig into why

I think this is the reason why it didn't improve on upgrade

https://github.com/civicrm/civicrm-core/pull/8719/files

Hopefully I can get some feedback on that in the next day or 2. Otherwise I think we might just deploy that patch to our live since it can't really make it worse

OK - it seems that

  1. it's intentional that if the cron is enabled & the versionCheck fails it falls back to a poor man's cron (the slooooowww situation)
  2. the logic seems to fail to kick in if versionCheck cron task is disabled
  3. so I've disabled the scheduled job - lets see if that is better tomorrow

NB if that does speed it up it's not a permanent fix - it's just taking advantage of someone's mistake but hopefully when they fix it they will do it in a way that doesn't hurt us

@RLewis @LeanneS Eileen has made some changes that may temporarily make Civi load faster. Can you let us know if you notice a difference?

After some discussion it seems that turning off the scheduled job is the right fix & should keep working - however unintuitive I found it!

Change 301023 had a related patch set uploaded (by Eileen):
Hack out version & extension checks.

https://gerrit.wikimedia.org/r/301023

Change 301023 merged by jenkins-bot:
Hack out version & extension checks.

https://gerrit.wikimedia.org/r/301023

@CaitVirtue can we close this out? I think there is still a lag resolving certificates but the big lag is much better now

@yeah, i guess. It's better but still could be improved. not sure how to include that in the backlog?

Shall we add a second ticket & close this? I tried just now & I experienced a small delay loading the page in the first place but no noticeable delay actually logging in. It seemed a bit less of a delay than I had noticed on other occasions - which makes me wonder if there is something else involved.

DStrine closed this task as Resolved.Sep 13 2016, 10:08 PM
mmodell removed a subscriber: awight.Jun 22 2017, 9:44 PM

Change 410381 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm/civicrm@master] Hack out version & extension checks.

https://gerrit.wikimedia.org/r/410381

Change 410381 merged by jenkins-bot:
[wikimedia/fundraising/crm/civicrm@master] Hack out version & extension checks.

https://gerrit.wikimedia.org/r/410381