User Details
- User Since
- Aug 21 2020, 11:05 AM (325 w, 1 d)
- Availability
- Available
- IRC Nick
- urbanecm
- LDAP User
- Urbanecm work
- MediaWiki User
- Martin Urbanec (WMF) [ Global Accounts ]
Wed, Nov 11
I discussed the above with @Michael. The overall conclusion was "given the amount of queries went down sufficiently, let's aim for whatever allows us to simplify the code" and prioritize maintainability.
Tue, Nov 10
For the record, --attach does display the output in full (in the same way as seen on beta). Maybe the -t simulates the terminal well?
Changing the output is not a problem – I'm mostly concerned that the fact "half an input isn't displayed" wasn't obvious (and it made me debug "why is the script only printing half of its output"). But, it might be something kubectl logs doesn't really expose to the caller. I understand this is an edge case though. Either way, thank you so much for writing detailed response! It is well appreciated.
The same happens on production as well:
FWIW, I haven't found anything easier than:
This is (to some extent) a regression, since the old database-backed locks were released when the connection got dropped.
urbanecm@deployment-deploy06:~$ mwscript GrowthExperiments:importArticleTopics --wiki=frwiki --topicSource=prod One of --count or --pageList is required urbanecm@deployment-deploy06:~$ mwscript GrowthExperiments:importArticleTopics --wiki=frwiki --topicSource=prod --count=20 --verbose Found 20 articles not found on enwiki: Accueil, Pack/2, Anastomose (biologie), Anatole Latuile (série télévisée d'animation), Calligraphie dans le monde islamique, Carte vitale du bâtiment Adding topics for Cigarette: History and Society.Business and economics=530, STEM.Medicine & Health=922, STEM.STEM*=971 Adding topics for Bad Doberan: Geography.Regions.Europe.Europe*=982, Geography.Regions.Europe.Western Europe=979 Adding topics for Conquête de l'Ouest: Geography.Regions.Americas.North America=985, History and Society.History=692 Adding topics for Berbérisme: Culture.Biography.Biography*=636, Geography.Regions.Africa.Africa*=998, Geography.Regions.Africa.Northern Africa=999 Adding topics for Carlo Michelstaedter: Culture.Philosophy and religion=890, Geography.Regions.Europe.Europe*=595, Geography.Regions.Europe.Southern Europe=528 Adding topics for Brownie Mary: Culture.Biography.Biography*=964, Culture.Biography.Women=855, Geography.Regions.Americas.North America=816, History and Society.History=703, STEM.Medicine & Health=959, STEM.STEM*=844 Adding topics for B.J. Tyler: Culture.Biography.Biography*=870, Culture.Sports=998, Geography.Regions.Americas.North America=713 Adding topics for Alessandro Frosini: Culture.Biography.Biography*=954, Culture.Sports=989, Geography.Regions.Europe.Europe*=894, Geography.Regions.Europe.Southern Europe=970 Adding topics for 2e arrondissement de Lyon: Geography.Regions.Europe.Europe*=987, Geography.Regions.Europe.Western Europe=988 Adding topics for Ali Mmadi: Culture.Biography.Biography*=949, Culture.Sports=989, Geography.Regions.Africa.Africa*=913, Geography.Regions.Europe.Europe*=503, Geography.Regions.Europe.Western Europe=845 Adding topics for Ataa Jaber: Culture.Biography.Biography*=983, Culture.Sports=999, Geography.Regions.Asia.Asia*=948, Geography.Regions.Asia.West Asia=998 Adding topics for Attentat de Yom Kippour à Halle-sur-Saale: Geography.Regions.Europe.Europe*=939, Geography.Regions.Europe.Western Europe=950 Adding topics for Arnold Temanfo: Culture.Biography.Biography*=961, Culture.Sports=939, Geography.Regions.Africa.Africa*=883, Geography.Regions.Africa.Central Africa=748 Adding topics for Marie-Claire Nseng-Elang: Culture.Biography.Biography*=976, Culture.Biography.Women=860, Geography.Regions.Africa.Africa*=991, Geography.Regions.Africa.Central Africa=999 Found 20 articles Found 20 articles not found on enwiki: Grand prix lycéen des compositeurs, Gustave Samanos, Henri Brocard (parfumeur), Henri Tribout de Morembert, Ingani, Iodopsine, Jean-Jacques Delfour, Jean-Pierre Vigato, Journée des Farines, Liste des gouverneurs actuels des provinces de la Guinée équatoriale, Liste des œuvres de Christian Jacq Adding topics for Jaurès Alferov: Culture.Biography.Biography*=950, History and Society.Society=904, STEM.STEM*=956 Adding topics for Indice de confiance des consommateurs: History and Society.Business and economics=987, STEM.Mathematics=576, STEM.STEM*=868 Adding topics for Jeu de lames: History and Society.Society=559, STEM.Biology=905, STEM.STEM*=986 Adding topics for Hérésie des Judaïsants: Culture.Philosophy and religion=799, Geography.Regions.Asia.Asia*=521, Geography.Regions.Asia.North Asia=618, Geography.Regions.Europe.Eastern Europe=553, History and Society.History=650, History and Society.Society=960 Adding topics for Lionel Daunais: Culture.Biography.Biography*=952, Culture.Media.Media*=938, Culture.Media.Music=992, Culture.Performing arts=704, Geography.Regions.Americas.North America=562 Adding topics for Herman Autrey: Culture.Biography.Biography*=991, Culture.Media.Media*=899, Culture.Media.Music=970 Adding topics for Kały-Towarzystwo: Geography.Geographical=925, Geography.Regions.Europe.Eastern Europe=999, Geography.Regions.Europe.Europe*=994 Adding topics for Jean-Charles Baquoy: Culture.Biography.Biography*=759, Geography.Regions.Europe.Europe*=939, Geography.Regions.Europe.Western Europe=981 Adding topics for Limbourg (fromage): Culture.Food and drink=997, Geography.Regions.Europe.Europe*=582, Geography.Regions.Europe.Western Europe=522 urbanecm@deployment-deploy06:~$
There's last remaining patch in this task, then we can call it a day. Moving to CR.
This should now be testable at all beta wikis.
Note as I just run into it on beta: If you want to enable PersonalDashboard everywhere, be sure to exclude the lockeddown dblist. https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1368346 is the beta fix.
Mon, Nov 9
- enwiki successfully finished as `mw-script.codfw.o0nrvt4b
- itwiki successfully finished as mw-script.codfw.xn81feo0
- eswiki successfully finished as mw-script.codfw.yp2t05dw
- frwiki is still running as mw-script.codfw.cq0subfv
- the all-wiki job is running as mw-script.codfw.cpk1hwxf; currently at jawiki
Sun, Nov 8
I looked at this again as part of the perf audit in T443778. Here is where we are.
1. A page view can start uncached searches and a full impact computation
- Path: includes/HomepageHooks.php:357 → includes/NewcomerTasks/TaskType/TaskTypeManager.php:92 → suggestNewTaskTypeForUser() → includes/LevelingUp/LevelingUpManager.php:245.
- Trigger: the user's only selected task type is link-recommendation, and the user has reached the Add Link edit limit.
- Cost on each page view for these users:
- One uncached getUserImpact().
- One search for each candidate task type, with useCache => false.
- The docblock of that method says to avoid it in the critical path.
- Fix: cache the suggested task type per user, or compute it in a job.
Thu, Nov 5
- The only reason CommunityConfiguration currently needs hCaptcha handling is that we run runEditFilterMergedContentHook to support AbuseFilter.
- Other stores wouldn't require a captcha automatically.
- Any CC edit caught with an "always require CAPTCHA" consequence AbuseFilter would currently fail (as we do not support captcha, and AbuseFilter overrides skipcaptcha)
- One way we could do this is to make stores indicate whether they require a captcha (and with what settings), and then the configuration form could read that. This is similar to how we currently handle access control.
- Another way would be to introduce a new CAPTCHA trigger (specific for CommunityConfiguration), and require CAPTCHA on any configuration edits coming from CommunityConfiguration. This would handle both edits as well as any store backends in the future.
- This is needed to continue supporting the decoupling of storage backends and the editing forms.
- PoC is available at https://gerrit.wikimedia.org/r/1364439.
- Support in ManageMentors would benefit from using more standard ways to create objects. Uploaded https://gerrit.wikimedia.org/r/1364449 to support it.
Wed, Nov 4
Excluding the file is probably not the best fix, but it gets the job done.
Critical, as it blocks libup: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ImpactModule/+/1362686
Related to legacy community configuration.
Statsd was removed, this is no longer needed.
Related to instrumentation that was eventually removed.
Our custom logger was removed, so this is no longer actionable.
