Page MenuHomePhabricator

Performance survey died on ruwiki on Sep 26
Closed, ResolvedPublic

Description

Since it only stopped on that wiki, I suspect that it was blocked by the community via site CSS or similar.

Details

Related Gerrit Patches:
operations/mediawiki-config : masterDefine performance survey in a more bulletproof way

Event Timeline

Gilles created this task.Oct 7 2019, 7:52 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 7 2019, 7:52 PM
Gilles updated the task description. (Show Details)Oct 7 2019, 7:53 PM
Gilles claimed this task.Oct 7 2019, 7:55 PM
Gilles triaged this task as Normal priority.
Gilles moved this task from Inbox to Doing on the Performance-Team board.
Gilles added a subscriber: stjn.Thu, Oct 24, 8:53 AM

@stjn did the ruwiki community block the survey on Sep 26? If you have a link to the corresponding community discussion for posterity, that would be helpful. There's no need to force the survey down people's throats as I'm not actively working on it. It was a nice-to-have to get an ongoing human performance metric, but I understand if people collectively got sick of seeing it.

I don't see anything related in Common.js/Common.css so it might not be what happened.

Gilles added a subscriber: Isaac.EditedThu, Oct 24, 9:00 AM

Ok, found the issue, @Isaac mistakenly overrode the default survey definition for ruwiki by rolling out another survey there: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/539183/

The "default" section of config maps is overwritten by wiki-specific ones.

Change 545779 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/mediawiki-config@master] Define performance survey in a more bulletproof way

https://gerrit.wikimedia.org/r/545779

Change 545779 merged by jenkins-bot:
[operations/mediawiki-config@master] Define performance survey in a more bulletproof way

https://gerrit.wikimedia.org/r/545779

Mentioned in SAL (#wikimedia-operations) [2019-10-24T09:12:16Z] <gilles@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T234853 Re-enable performance perception survey on ruwiki (duration: 01m 04s)

I'm still not seeing it in mw.config.get( 'wgEnabledQuickSurveys' ) on ruwiki, I don't know what's up with that. Trying some things out on mwmaint1002, it should pick it up.

It's as if the startup module isn't picking up the config change that was deployed. I briefly got a version of the startup module that had it by requesting its URL directly:

Then it was gone again!

~/Downloads $ curl -s "https://ru.wikipedia.org/w/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector" | grep perceived
~/Downloads $

https://ru.wikipedia.org/w/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector

@Krinkle any ideas?

It's as if some app servers got the new config and others not, and it's playing russian roulette every time the startup module is re-generated. The deployment was successful, though:

sync-proxies: 100% (ok: 9; fail: 0; left: 0)                                    
09:12:02 Finished sync-proxies (duration: 00m 02s)
09:12:02 Started sync-apaches
sync-apaches: 100% (ok: 311; fail: 0; left: 0)                                  
09:12:09 Finished sync-apaches (duration: 00m 07s)
09:12:09 Check php-fpm cache...
09:12:15 Synchronized wmf-config/InitialiseSettings.php: T234853 Re-enable performance perception survey on ruwiki (duration: 01m 04s)

Yep, confirmed, I see the config now that the startup module has been generated by another server. I've checked on one of the misbehaving app servers and the file on the FS is up to date. I suspect this has to do with PHP not picking up the new version of the file.

stjn removed a subscriber: stjn.Thu, Oct 24, 10:21 AM
Isaac added a comment.Thu, Oct 24, 1:29 PM

Ahhh apologies @Gilles -- I did not realize think anything of those surveys because the coverage was set at 0. It looks like you have made them more explicit to avoid someone like me making this error in the future but anything else I should be aware of when deploying surveys in the future? There is the hope that we will deploy a few in Hindi (hi), bahasa Indonesian (id), and Portuguese (pt) in the next month or so.

Gilles closed this task as Resolved.Thu, Oct 24, 1:52 PM

No worries, it happens. Just keep that principle in mind for MediaWiki config in general, wiki-specific replaces default, it's not adding to it. I don't think there's any other big gotcha for surveys.

Isaac added a comment.Thu, Oct 24, 2:24 PM

Thanks, will do -- I appreciate you being part of my learning curve!