Page MenuHomePhabricator

Improve preference migration script
Closed, ResolvedPublic

Description

Currently, preference migration script (https://phabricator.wikimedia.org/diffusion/EULS/browse/master/maintenance/ULSCompactLinksDisablePref.php) takes long time to perform run on large wiki (enwiki in this case). Update and improve script, if needed to make run time shorter.

Also see: https://phabricator.wikimedia.org/T133031

Event Timeline

Don't join with the revision table, that is what is causing the slowdown.

The script is only going to be used one more time. Even though it is slow, that can be better option than spend more time on fixing it before we throw it away.

Also, per my understanding, even though the part about collecting users is slow, the part about updating their preference is a magnitude slower and improving the first part wouldn't help much.

jcrespo added a comment.EditedFeb 21 2018, 1:26 PM

If you want to run many inefficient, select * from revision queries on production, that is cool to me, but don't run them on a random slave, we have 'slow' slaves for that.

jcrespo added a subscriber: greg.Feb 21 2018, 3:03 PM

@greg I would like the deployment of this maintanance script blocked until it uses a 'vslow' replica for the inneficient reads, to avoid affecting the other servers- that is literally a 9-characters patch, I think it is not asking too much.

Change 413173 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/extensions/UniversalLanguageSelector@master] Use a replica dedicated to slow queries (if available)

https://gerrit.wikimedia.org/r/413173

KartikMistry triaged this task as High priority.Feb 21 2018, 3:44 PM
KartikMistry moved this task from Backlog to In Review on the Language-2018-Jan-Mar board.

@jcrespo Is it OK to go ahead after https://gerrit.wikimedia.org/r/413173 is merged and deployed? Please let us know if there any other blockers remaining.

Yes, that was my only blocker- the other were just suggestions because you seem worried about the time length.

Change 413173 merged by jenkins-bot:
[mediawiki/extensions/UniversalLanguageSelector@master] Use a replica dedicated to slow queries (if available)

https://gerrit.wikimedia.org/r/413173

@Nikerabbit - I have some questions:
(1) I'd like to run the script in betalabs for some of my test users, but the only usage options that I saw was

Usage: mwscript scriptName.php --wiki=dbname

Does the scirpt run only for an entire wiki or I can specify a user or a range of users?

(2) betalabs has CLL only as a beta feature. Is it intentional? cx-testing and testwiki (wmf.22) has CLL as an option in Preferences-Appearance.

Change 414608 had a related patch set uploaded (by KartikMistry; owner: Nikerabbit):
[mediawiki/extensions/UniversalLanguageSelector@wmf/1.31.0-wmf.22] Use a replica dedicated to slow queries (if available)

https://gerrit.wikimedia.org/r/414608

@Nikerabbit - I have some questions:
(1) I'd like to run the script in betalabs for some of my test users, but the only usage options that I saw was

Usage: mwscript scriptName.php --wiki=dbname

Does the scirpt run only for an entire wiki or I can specify a user or a range of users?

It will run on entire wiki by default. With --continue option you can specify user ID and it will start from there.

(2) betalabs has CLL only as a beta feature. Is it intentional? cx-testing and testwiki (wmf.22) has CLL as an option in Preferences-Appearance.

CLL in Production is out beta in all wikis except enwiki as of now. Probably, it should be out of Beta in Betalabs. I'll look into it.

Change 414608 merged by jenkins-bot:
[mediawiki/extensions/UniversalLanguageSelector@wmf/1.31.0-wmf.22] Use a replica dedicated to slow queries (if available)

https://gerrit.wikimedia.org/r/414608

Mentioned in SAL (#wikimedia-operations) [2018-02-26T14:10:38Z] <zfilipin@tin> Synchronized php-1.31.0-wmf.22/extensions/UniversalLanguageSelector/maintenance/ULSCompactLinksDisablePref.php: SWAT: [[gerrit:414609|Added option to continue script from particular User ID]] [[gerrit:414608|Use a replica dedicated to slow queries (if available) (T187880)]] (duration: 00m 58s)

(2) betalabs has CLL only as a beta feature. Is it intentional? cx-testing and testwiki (wmf.22) has CLL as an option in Preferences-Appearance.

CLL in Production is out beta in all wikis except enwiki as of now. Probably, it should be out of Beta in Betalabs. I'll look into it.

Once we have CLL out of beta in enwiki, this should be fixed. You can check in: https://es.wikipedia.beta.wmflabs.org/wiki/Especial:Preferencias#mw-prefsection-betafeatures where CLL is out of beta.

Thanks, @KartikMistry!
Checked in enwiki - the beta option is moved to Preferences-Appearance and CLL functionality is fine. It is a default option - enabled when a user clicks on "Restore all default settings..." cx-testing behaves as the production.

One small note: the preference "Use a compact language list..." is not a default in betalabs - " "Restore all default settings..." will not make it enabled.

Etonkovidova moved this task from QA to Done on the Language-2018-Jan-Mar board.Feb 28 2018, 7:33 PM

The script is improved.

The issue in beta is probably separate.

Can this be closed?

Nikerabbit closed this task as Resolved.Mar 4 2018, 6:26 PM

Yes.

Can the script be deleted now?

Well... It's still beta on some sites and we may want to run it again. So not yet. We'll give it back thought soon enough