Page MenuHomePhabricator

Ladsgroup (Amir Sarabadani (WMDE))
Shah of BugsAdministrator

Projects (29)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Sunday

  • Clear sailing ahead.

User Details

User Since
Oct 6 2014, 9:53 PM (295 w, 4 d)
Roles
Administrator
Availability
Available
IRC Nick
Amir1
LDAP User
Ladsgroup
MediaWiki User
Ladsgroup [ Global Accounts ]

Wikidata software engineer, open source enthusiast, mediawiki volunteer developer, long-term Wikipedian
Babel: fa-N, en-4, de-2, tr-1, hu-1

Recent Activity

Today

Ladsgroup moved T254536: CacheAwarePropertyInfoStore performs 4000 Memc ops/s (APC not working?) from To Do to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Fri, Jun 5, 4:55 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Sustainability (Incident Prevention), Wikidata
Ladsgroup edited projects for T254536: CacheAwarePropertyInfoStore performs 4000 Memc ops/s (APC not working?), added: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞); removed Wikidata-Campsite.

This is basically a ticking bomb

Fri, Jun 5, 4:55 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Sustainability (Incident Prevention), Wikidata
Ladsgroup added a comment to T154601: Grafana: "wikidata-datamodel-terms" doesn't update anymore.

I won't be around but also I want to mention I'm not sure reportupdater would be a great idea here. We are not using the codebase, reportupdater is not under active development (44 commits according to github), its yaml files are scattered everywhere, we have our own system of refinery scripts that we can use.

Fri, Jun 5, 4:26 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-GoranSMilovanovic, User-Addshore, WMDE-Analytics-Engineering, Wikidata
Ladsgroup added a comment to T111853: The href of be-tarask: interlanguage link points to the be-x-old domain.

Could we remove be-x-old from langlist? Can it break something?

Fri, Jun 5, 1:25 PM · User-Ladsgroup, MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Wikimedia-Interwiki-links, WMF-deploy-2015-09-08_(1.26wmf22), WMF-deploy-2015-09-15_(1.26wmf23), WMF-deploy-2015-09-01_(1.26wmf21), Patch-For-Review, MediaWiki-extensions-WikimediaMaintenance, Wikimedia-Site-requests
Ladsgroup added a comment to T111853: The href of be-tarask: interlanguage link points to the be-x-old domain.

Okay, It should be fixed now.
My suggestion is to add be-tarask to langlist file. That solved the issue and generated the hotfix I just deployed. the hotfix will get undeployed if 602675 doesn't get merged soon (and someone runs the scap update-interwiki-cache) . @Urbanecm Having the language name in langlist file seems less hacky than put it into interwiki map. What do you think?

Fri, Jun 5, 12:58 PM · User-Ladsgroup, MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Wikimedia-Interwiki-links, WMF-deploy-2015-09-08_(1.26wmf22), WMF-deploy-2015-09-15_(1.26wmf23), WMF-deploy-2015-09-01_(1.26wmf21), Patch-For-Review, MediaWiki-extensions-WikimediaMaintenance, Wikimedia-Site-requests
Ladsgroup added a comment to T111853: The href of be-tarask: interlanguage link points to the be-x-old domain.

Looking at the logic of dumpInterwiki.php, it uses the db name, which would explain why but that might be the reason, let me dig deeper and find a solution.

Fri, Jun 5, 12:28 PM · User-Ladsgroup, MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Wikimedia-Interwiki-links, WMF-deploy-2015-09-08_(1.26wmf22), WMF-deploy-2015-09-15_(1.26wmf23), WMF-deploy-2015-09-01_(1.26wmf21), Patch-For-Review, MediaWiki-extensions-WikimediaMaintenance, Wikimedia-Site-requests
Ladsgroup added a comment to T111853: The href of be-tarask: interlanguage link points to the be-x-old domain.

@Ladsgroup It broke be-tarask interwiki at all. I think this might have something in common with db being named be_x_oldwiki. Could we rename db (and thus fix be-tarask interwiki)?

Otherwise we had to revert the double-redirect patch. Or add another be-x-old -> be-tarask redirect into MW config to fix T111853

Fri, Jun 5, 12:24 PM · User-Ladsgroup, MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Wikimedia-Interwiki-links, WMF-deploy-2015-09-08_(1.26wmf22), WMF-deploy-2015-09-15_(1.26wmf23), WMF-deploy-2015-09-01_(1.26wmf21), Patch-For-Review, MediaWiki-extensions-WikimediaMaintenance, Wikimedia-Site-requests
Ladsgroup added a comment to T254536: CacheAwarePropertyInfoStore performs 4000 Memc ops/s (APC not working?).

I was wrong, we somehow removed the APCu cache bit altogether, I should find out what happened.

Fri, Jun 5, 11:42 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Sustainability (Incident Prevention), Wikidata
Ladsgroup added a comment to T254536: CacheAwarePropertyInfoStore performs 4000 Memc ops/s (APC not working?).

I think I found out what's wrong. The APCu cache seems to be per-wiki but this doesn't need to be like that. Let me fix it.

Fri, Jun 5, 11:26 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Sustainability (Incident Prevention), Wikidata
Ladsgroup added a comment to T244340: Reduce read pressure on memcached servers by adding a machine-local Memcache instance.

Seconding Krinkle. I investigate more on this. I have lots of ideas on how to improve memcached read pressure (we load lots of items from memcached sometimes for example) but I need to have more metrics on what keys are being accessed too many times, what are the large data that is being sent often, what requests caused lots of read, etc. If it's not possible, it's fine, just let me know.

Fri, Jun 5, 11:16 AM · Performance-Team, Patch-For-Review, Operations, serviceops
Ladsgroup added a comment to T254536: CacheAwarePropertyInfoStore performs 4000 Memc ops/s (APC not working?).

This really really shouldn't happen. I investigate.

Fri, Jun 5, 11:10 AM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, Sustainability (Incident Prevention), Wikidata
Ladsgroup closed T111853: The href of be-tarask: interlanguage link points to the be-x-old domain, a subtask of T113461: pywikibot.Link should support interwiki prefix be-x-old, as Resolved.
Fri, Jun 5, 11:05 AM · Pywikibot
Ladsgroup closed T111853: The href of be-tarask: interlanguage link points to the be-x-old domain, a subtask of T145535: language code disparity (Wikidata tracking), as Resolved.
Fri, Jun 5, 11:05 AM · I18n, Tracking-Neverending, Wikidata
Ladsgroup closed T111853: The href of be-tarask: interlanguage link points to the be-x-old domain as Resolved.

Done: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/602652/1/wmf-config/interwiki.php

Fri, Jun 5, 11:05 AM · User-Ladsgroup, MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Wikimedia-Interwiki-links, WMF-deploy-2015-09-08_(1.26wmf22), WMF-deploy-2015-09-15_(1.26wmf23), WMF-deploy-2015-09-01_(1.26wmf21), Patch-For-Review, MediaWiki-extensions-WikimediaMaintenance, Wikimedia-Site-requests

Yesterday

Ladsgroup added a comment to T253291: Deploy Growth features on Persian Wikipedia.

I assume it's good to go now? Shall we deploy soon?

Thu, Jun 4, 9:19 PM · MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Growth-Team
Ladsgroup updated the task description for T253291: Deploy Growth features on Persian Wikipedia.
Thu, Jun 4, 9:19 PM · MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Growth-Team
Ladsgroup added a comment to T253812: Correct the directionality and contents of the textbox placeholders in New Item form.

Why not show a completely localized sentence? We already have the messages for it.

Thu, Jun 4, 9:03 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), I18n, MediaWiki-extensions-WikibaseRepository, Wikidata
Ladsgroup added a comment to T253812: Correct the directionality and contents of the textbox placeholders in New Item form.

Why bidi is so fun?
I looked at a little. We can use ::placeholder pseudo-element while it's not supported in IE: https://caniuse.com/#search=%3A%3Aplaceholder

Thu, Jun 4, 8:40 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), I18n, MediaWiki-extensions-WikibaseRepository, Wikidata
Ladsgroup claimed T235420: Create wikidata badges to indicate when sitelinks point to Wikipedia redirect pages.
Thu, Jun 4, 8:21 PM · User-Ladsgroup, Patch-For-Review, MW-1.35-notes (1.35.0-wmf.20; 2020-02-18), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Ladsgroup moved T251876: wbsgetsuggestions RevisionedUnresolvedRedirectException logspam from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Thu, Jun 4, 8:20 PM · MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata
Ladsgroup moved T253737: Release and use wikibase/data-model 9.5.1 everywhere from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Thu, Jun 4, 8:20 PM · MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata, Wikibase-DataModel, PHP 7.4 support
Ladsgroup added a project to T65732: [Story] Handle localised digits in formatters and parsers: Wikidata-Campsite.
Thu, Jun 4, 8:16 PM · Wikidata-Campsite, Story, Wikidata, patch-welcome, MediaWiki-extensions-WikibaseRepository
Ladsgroup added a comment to T235420: Create wikidata badges to indicate when sitelinks point to Wikipedia redirect pages.

I set up a set of patches that basically allows redirects to be set in wikidata as sitelink (whether if they have a badge or not). Is that acceptable @Lydia_Pintscher ?

Thu, Jun 4, 4:36 PM · User-Ladsgroup, Patch-For-Review, MW-1.35-notes (1.35.0-wmf.20; 2020-02-18), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Ladsgroup added a comment to T251880: Special:Search RevisionedUnresolvedRedirectException logspam.

I can't reproduce this. Got fixed automagically?

Thu, Jun 4, 2:11 PM · Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, Wikidata
Ladsgroup added a comment to T237164: Wikibase's cache/integration-tests incompatible with PHPUnit 8.

which option shall we pursue? Any preference?

Thu, Jun 4, 12:55 PM · MW-1.35-notes (1.35.0-wmf.20; 2020-02-18), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, MediaWiki-extensions-WikibaseRepository
Ladsgroup added a comment to T253461: Liberate the @ for AtEase.

My two cents:

  • Has any sort of profiling done on @ and AtEase, one might be quite faster. We should check.
  • I'm always supportive of reducing code base that we need to maintain, one less repository to maintain, no matter how small, is potential for more work in critical areas for us
Thu, Jun 4, 9:58 AM · TechCom, MediaWiki-Codesniffer, Proposal, at-ease, MediaWiki-General
Ladsgroup moved T254387: SpecialSetSiteLinkTest::testExecutePostRemoveSiteLink & testExecutePostModifySiteLink failing for no apparent reason from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Thu, Jun 4, 9:42 AM · MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
toan awarded T254387: SpecialSetSiteLinkTest::testExecutePostRemoveSiteLink & testExecutePostModifySiteLink failing for no apparent reason a Yellow Medal token.
Thu, Jun 4, 7:09 AM · MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Pablo-WMDE awarded T254387: SpecialSetSiteLinkTest::testExecutePostRemoveSiteLink & testExecutePostModifySiteLink failing for no apparent reason a Stroopwafel token.
Thu, Jun 4, 7:08 AM · MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata

Wed, Jun 3

Ladsgroup added a comment to T253291: Deploy Growth features on Persian Wikipedia.
Wed, Jun 3, 10:36 PM · MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Growth-Team
Ladsgroup edited P11391 generate MediaWiki:NewcomerTasks.
Wed, Jun 3, 10:35 PM
Ladsgroup created P11391 generate MediaWiki:NewcomerTasks.
Wed, Jun 3, 10:33 PM
Ladsgroup added a comment to T244001: Investigate reminding people to update `npm audit`.

@Addshore Maybe I'm missing something obvious but looking at https://gerrit.wikimedia.org/r/c/integration/config/+/573235/3/jjb/job-templates.yaml it does only checks the main wikibase package.json and not the important ones like bridge, TR, termbox (honestly, running npm audit on these submodules exploded majestically)

Wed, Jun 3, 9:43 PM · User-Addshore, Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Wikidata-Tainted-References-Sprint10, Wikidata-Tainted-References-Sprint9
Ladsgroup added a comment to T244001: Investigate reminding people to update `npm audit`.

The most recent run says these two need updating:

# Run  npm install --save-dev stylelint-config-wikimedia@0.10.1  to resolve 1 vulnerability
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ Low           │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ yargs-parser                                                 │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ stylelint-config-wikimedia [dev]                             │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ stylelint-config-wikimedia > stylelint > meow > yargs-parser │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://nodesecurity.io/advisories/1500                      │
└───────────────┴──────────────────────────────────────────────────────────────┘
Wed, Jun 3, 9:18 PM · User-Addshore, Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata, Wikidata-Tainted-References-Sprint10, Wikidata-Tainted-References-Sprint9
Ladsgroup claimed T253722: Convert some properties string->external ID.
ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P1150 --new-data-type external-id
Successfully updated the property data type to external-id.
ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P1161 --new-data-type external-id
Successfully updated the property data type to external-id.
ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P1162 --new-data-type external-id
Successfully updated the property data type to external-id.
ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P5858 --new-data-type external-id
Successfully updated the property data type to external-id.
ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P1036 --new-data-type external-id
Successfully updated the property data type to external-id.
ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P1149 --new-data-type external-id
Successfully updated the property data type to external-id.
ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P1190 --new-data-type external-id
Successfully updated the property data type to external-id.
ladsgroup@mwmaint1002:~$
Wed, Jun 3, 9:14 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Ladsgroup moved T254387: SpecialSetSiteLinkTest::testExecutePostRemoveSiteLink & testExecutePostModifySiteLink failing for no apparent reason from To Do to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Wed, Jun 3, 9:03 PM · MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Ladsgroup claimed T254387: SpecialSetSiteLinkTest::testExecutePostRemoveSiteLink & testExecutePostModifySiteLink failing for no apparent reason.

Let me take a look at this.

Wed, Jun 3, 8:37 PM · MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Ladsgroup added a comment to T253802: Set wgCheckUserLogLogins to true on WMF wikis to log successful and unsuccessful login attempts in CheckUser.

For bots, although they sometimes login more frequently than human users, that is not the majority of them. Bots that make significant numbers of edits usually login only once in a while. So their footprint should be small.

Wed, Jun 3, 8:22 PM · DBA, Trust-and-Safety, WMF-Legal, Patch-For-Review, CheckUser, Wikimedia-Site-requests
Ladsgroup added a comment to T249563: Hard deprecate WikiPage::doEditUpdates.

Okay, Wikibase can parse some old serialization formats. But I have no idea if this is also true for whatever serialization format we used back in 2012, because it also seems plausible to me that we would have relied on the rebuild script until then, and only started to support old serialization formats from after that point.

Wed, Jun 3, 8:16 PM · Wikidata-Campsite, Wikidata, MediaWiki-extensions-WikibaseRepository, Core Platform Team Workboards (Clinic Duty Team), Technical-Debt (Deprecation process), MediaWiki-Revision-backend, User-DannyS712
Ladsgroup removed a project from T252841: Update the method Wikibase uses to inject edit language links into the language portal: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞).

I'm removing wikidata tag as currently there's nothing for the team to review. Re-add the tag if we can review any patch.

Wed, Jun 3, 1:42 PM · MW-1.35-release, MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Wikidata, Patch-For-Review, Vector (Vector (Tracking)), Readers-Web-Backlog (Tracking), User-Jdlrobson
Ladsgroup moved T254114: Make it possible to indent properties in MediaWiki:Wikibase-SortedProperties from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Wed, Jun 3, 1:39 PM · MW-1.35-notes (1.35.0-wmf.36; 2020-06-09), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-extensions-WikibaseRepository, User-DannyS712, Wikidata
Ladsgroup added a comment to T249563: Hard deprecate WikiPage::doEditUpdates.

Likely, but I don’t know if we want to support upgrading old wikibases or not (if yes, we might need a replacement for that method). @Addshore, do you know?

Wed, Jun 3, 1:28 PM · Wikidata-Campsite, Wikidata, MediaWiki-extensions-WikibaseRepository, Core Platform Team Workboards (Clinic Duty Team), Technical-Debt (Deprecation process), MediaWiki-Revision-backend, User-DannyS712
Ladsgroup added a comment to T253802: Set wgCheckUserLogLogins to true on WMF wikis to log successful and unsuccessful login attempts in CheckUser.

login notify is not great, it only counts login from a new device or unsuccessful login. Looking at logstash, the number is massive, a couple of times larger. And it's mostly bots, sometimes they login several times a minute continuously (which wouldn't be captured in login notify). Unsuccessful logins are small though.

Wed, Jun 3, 11:25 AM · DBA, Trust-and-Safety, WMF-Legal, Patch-For-Review, CheckUser, Wikimedia-Site-requests
Ladsgroup added a comment to T253802: Set wgCheckUserLogLogins to true on WMF wikis to log successful and unsuccessful login attempts in CheckUser.

Looking at grafana for login notify, enabling this would add more than 200 rows per minute in total, Assuming the cu_changes table keeps data for 90 days. it'll be more than 26M rows.

Wed, Jun 3, 11:07 AM · DBA, Trust-and-Safety, WMF-Legal, Patch-For-Review, CheckUser, Wikimedia-Site-requests
Ladsgroup updated the task description for T65732: [Story] Handle localised digits in formatters and parsers.
Wed, Jun 3, 11:01 AM · Wikidata-Campsite, Story, Wikidata, patch-welcome, MediaWiki-extensions-WikibaseRepository
Ladsgroup added a comment to T65732: [Story] Handle localised digits in formatters and parsers.

I'm pretty sure there are more tickets. I think I also filed one.

Wed, Jun 3, 10:59 AM · Wikidata-Campsite, Story, Wikidata, patch-welcome, MediaWiki-extensions-WikibaseRepository
Ladsgroup merged task T254273: Some parts of dates on Wikidata are shown using non-localized digits into T65732: [Story] Handle localised digits in formatters and parsers.
Wed, Jun 3, 10:58 AM · I18n, MediaWiki-extensions-WikibaseClient, Wikidata
Ladsgroup merged T254273: Some parts of dates on Wikidata are shown using non-localized digits into T65732: [Story] Handle localised digits in formatters and parsers.
Wed, Jun 3, 10:58 AM · Wikidata-Campsite, Story, Wikidata, patch-welcome, MediaWiki-extensions-WikibaseRepository

Tue, Jun 2

Ladsgroup added a comment to T253802: Set wgCheckUserLogLogins to true on WMF wikis to log successful and unsuccessful login attempts in CheckUser.

Has this been approved by Legal, T&S and DBA? I think we should get it before moving forward.

Tue, Jun 2, 11:54 AM · DBA, Trust-and-Safety, WMF-Legal, Patch-For-Review, CheckUser, Wikimedia-Site-requests
Ladsgroup added a comment to T119173: RFC: Discourage use of MySQL's ENUM type.

I see, you are constructing the class directly. I wonder how this is going to work with the new stable interface policy, since if it doesn't get annotated with @newable for the 1.35 release, by strict reading of the policy, there is no guarantee it won't be changed in the future without deprecation.

Tue, Jun 2, 11:22 AM · Performance-Team (Radar), TechCom-RFC, MediaWiki-General, DBA
Ladsgroup moved T251518: Integration tests for Pipe 2 from Peer Review to Done on the Wikidata - Reference Hunt - Sprint 6 board.
Tue, Jun 2, 11:01 AM · Wikidata - Reference Hunt - Sprint 6, User-Ladsgroup, Wikidata
Ladsgroup added a comment to T119173: RFC: Discourage use of MySQL's ENUM type.

Clarification: I think we need to decide whether we need another table or hardcoded numbers in the code on case-by-case basis but we should not use enum for any of those.

Tue, Jun 2, 11:01 AM · Performance-Team (Radar), TechCom-RFC, MediaWiki-General, DBA
Ladsgroup added a comment to T119173: RFC: Discourage use of MySQL's ENUM type.

NameTableStore, added in 1.31, provides a convenient way of handling pseudo-enums from the PHP side.

As far as I can see, only core code can use it – it doesn't have any extension points for extensions.

Tue, Jun 2, 10:24 AM · Performance-Team (Radar), TechCom-RFC, MediaWiki-General, DBA

Sat, May 30

Ladsgroup added a comment to T244392: [GOAL] Build and deploy the new Vue.js search experience.

I just wanted to say as a Wikipedian for 15 years, every time I see the design in the top of the ticket, my heart melts. Great work. Keep it up <3

Sat, May 30, 10:17 PM · Goal, Vue.js (Vue.js-Search), Readers-Web-Backlog (Kanbanana-2019-20-Q4), Desktop Improvements
Ladsgroup awarded T244392: [GOAL] Build and deploy the new Vue.js search experience a Love token.
Sat, May 30, 7:43 PM · Goal, Vue.js (Vue.js-Search), Readers-Web-Backlog (Kanbanana-2019-20-Q4), Desktop Improvements
Ladsgroup claimed T253439: Eliminate the toil in WMF wiki creation.
Sat, May 30, 3:09 PM · User-Ladsgroup, Wikimedia-General-or-Unknown
Ladsgroup updated the task description for T253439: Eliminate the toil in WMF wiki creation.
Sat, May 30, 3:09 PM · User-Ladsgroup, Wikimedia-General-or-Unknown
Ladsgroup added a comment to T253439: Eliminate the toil in WMF wiki creation.

Now the bot automatically adds and updates the description, here's an example: https://phabricator.wikimedia.org/transactions/detail/PHID-XACT-TASK-7yfvmpcvr7dcd5l/ it checks the open tickets of wiki creation project once a day. Do not touch the automatic text as it will get overridden or worse, if you remove the ending or beginning, the bot appends the whole thing again.

Sat, May 30, 3:08 PM · User-Ladsgroup, Wikimedia-General-or-Unknown
Ladsgroup updated the task description for T253029: Create Shan Wiktionary.
Sat, May 30, 2:49 PM · Patch-For-Review, MW-1.35-notes (1.35.0-wmf.34; 2020-05-26), User-Urbanecm, Wiki-Setup (Create)
Ladsgroup updated the task description for T246945: New Public Wiki for the API Portal.
Sat, May 30, 2:46 PM · Patch-For-Review, Release-Engineering-Team-TODO, CPT Initiatives (API Gateway), User-brennen, User-Ladsgroup, Wiki-Setup (Create), User-Urbanecm, Core Platform Team
Ladsgroup updated the task description for T253029: Create Shan Wiktionary.
Sat, May 30, 2:46 PM · Patch-For-Review, MW-1.35-notes (1.35.0-wmf.34; 2020-05-26), User-Urbanecm, Wiki-Setup (Create)
Ladsgroup added a comment to T246945: New Public Wiki for the API Portal.

okay, to recap:

  • The dns and puppet patches are blocker for me
  • The comments made by @Jdforrester-WMF hasn't been answered yet. I think this is really important as this can't be easily changed later. Either a fishbowl wiki or a public wiki connected to SUL. We might get away with a public wiki (everyone can create account) without being connected to SUL but as I said, these are really hard to fix later.
  • Nitpick: "API Portal" doesn't look right as project name, Do you mean "Wikimedia API Portal"? It's not that important as it can be changed later.
Sat, May 30, 2:28 PM · Patch-For-Review, Release-Engineering-Team-TODO, CPT Initiatives (API Gateway), User-brennen, User-Ladsgroup, Wiki-Setup (Create), User-Urbanecm, Core Platform Team
Ladsgroup updated the task description for T253029: Create Shan Wiktionary.
Sat, May 30, 2:10 PM · Patch-For-Review, MW-1.35-notes (1.35.0-wmf.34; 2020-05-26), User-Urbanecm, Wiki-Setup (Create)
Ladsgroup updated the task description for T253029: Create Shan Wiktionary.
Sat, May 30, 2:08 PM · Patch-For-Review, MW-1.35-notes (1.35.0-wmf.34; 2020-05-26), User-Urbanecm, Wiki-Setup (Create)
Ladsgroup updated the task description for T246945: New Public Wiki for the API Portal.
Sat, May 30, 2:05 PM · Patch-For-Review, Release-Engineering-Team-TODO, CPT Initiatives (API Gateway), User-brennen, User-Ladsgroup, Wiki-Setup (Create), User-Urbanecm, Core Platform Team
Ladsgroup added a comment to T253291: Deploy Growth features on Persian Wikipedia.

I still don't see the suggested edits section in beta cluster while database says I have it enabled:

MariaDB [fawiki]> select * from user_properties where up_user = 35 and up_property = 'growthexperiments-homepage-suggestededits-activated';
+---------+-----------------------------------------------------+----------+
| up_user | up_property                                         | up_value |
+---------+-----------------------------------------------------+----------+
|      35 | growthexperiments-homepage-suggestededits-activated | 1        |
+---------+-----------------------------------------------------+----------+
1 row in set (0.00 sec)
Sat, May 30, 1:42 PM · MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Growth-Team
Ladsgroup added a comment to T253291: Deploy Growth features on Persian Wikipedia.

I added the patch to make it use ores but also the problem was that the configuration pages don't exit yet. I created one but I will add more later.

Sat, May 30, 1:25 PM · MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Growth-Team
Ladsgroup added a comment to T253291: Deploy Growth features on Persian Wikipedia.

Okay, it's deployed in beta cluster. The only thing I haven't figured out yet is the configuration for suggested edits. I think "morelike" would work for now but I get this error in my console when trying it in my homepage:
Unable to load topic data for suggested edits: The configuration title does not exist.
After refresh, the whole thing is gone now :(

Sat, May 30, 12:34 PM · MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Growth-Team
Ladsgroup added a comment to T253291: Deploy Growth features on Persian Wikipedia.

okay, let's get this patch merged and deployed, once it's there, we can enable it in production. What do you think?

Sat, May 30, 12:03 PM · MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Growth-Team
Ladsgroup added a comment to T252091: RFC: Site-wide edit rate limiting with PoolCounter.

I hope you don't mind if I contradict my previous comment a bit, since my thinking is still evolving on this.

No worries at all. I'm also changing my mind quickly here.

One problem with using lag as the metric is that it doesn't go negative, so the integral will not be pulled down while the service is idle. We could subtract a target lag, say 1 minute, but that loses some of the supposed benefit of including an integral term. A better metric would be updater load, i.e. demand/capacity. When the load is more than 100%, the lag increases at a rate of 1 second per second, but there's no further information in there as to how heavily overloaded it is. When the load is less than 100%, lag decreases until it reaches zero. While it's decreasing, the slope tells you something about how underloaded it is, but once it hits zero, you lose that information.

Load is average queue size, if you take the currently running batch as being part of the queue. WDQS currently does not monitor the queue size. I gather (after an hour or so of research, I'm new to all this) that with some effort, KafkaPoller could obtain an estimate of the queue size by subtracting the current partition offsets from KafkaConsumer.endOffsets().

Failing that, we can make a rough approximation from available data. We can get the average utilisation of the importer from the rdf-repository-import-time-cnt metric. You can see in Grafana that the derivative of this metric hovers between 0 and 1 when WDQS is not lagged, and remains near 1 when WDQS is lagged. The metric I would propose is to add replication lag to this utilisation metric, appropriately scaled: utilisation + K_lag * lag - 1 where K_lag is say 1/60s. This is a metric which is -1 at idle, 0 when busy with no lag, and 1 with 1 minute of lag. The control system would adjust the request rate to keep this metric (and its integral) at zero.

With PID, we need to define three constants K_p, K_i and K_d. If we had problem with finding the pool size, this is going to get three times more complicated (I didn't find a standard way to determine these coefficients, maybe I'm missing something obvious)

One way to simplify it is with K_d=0, i.e. make it a PI controller. Having the derivative in there probably doesn't add much. Then it's only two times more complicated. Although I added K_lag so I suppose we are still at 3. The idea is that it shouldn't matter too much exactly what K_p and K_i are set to -- the system should be stable and have low lag with a wide range of parameter values. So you just pick some values and see if it works.

We currently don't have an infrastructure to hold the "maxlag" data over time so we can calculate its derivative and integral. Should we use redis? How it's going to look like? These are questions, I don't have answers for them. Do you have ideas for that?

WDQS lag is currently obtained by having an ApiMaxLagInfo hook handler which queries Prometheus, caching the result. Prometheus has a query language which can perform derivatives ("rate") and integrals ("sum_over_time") on metrics. So it would be the same system as now, just with a different Prometheus query.

Sat, May 30, 11:22 AM · Sustainability (Incident Prevention), User-Addshore, Wikidata-Campsite, Wikidata, TechCom-RFC
Ladsgroup moved T251271: Improve Storage Reading from Peer Review to Done on the Wikidata - Reference Hunt - Sprint 6 board.
Sat, May 30, 10:31 AM · User-Ladsgroup, Wikidata - Reference Hunt - Sprint 6, Wikidata

Fri, May 29

Ladsgroup awarded Blog Post: Celebrating 600,000 commits for Wikimedia a Party Time token.
Fri, May 29, 11:48 PM
Ladsgroup added a comment to T246945: New Public Wiki for the API Portal.

I need the DNS patch and the puppet patch be merged and deployed before I can continue. SRE team should take a look.

Fri, May 29, 9:12 AM · Patch-For-Review, Release-Engineering-Team-TODO, CPT Initiatives (API Gateway), User-brennen, User-Ladsgroup, Wiki-Setup (Create), User-Urbanecm, Core Platform Team

Thu, May 28

Ladsgroup added a comment to T253552: Detailed Reports from game DB.

Follow-up after our meeting @darthmon_wmde:

Please someone ping me when we have the data for this and let me know where do the data live.

Also, I can now see some sample data on https://tools.wmflabs.org/wd-ref-island/stats.php from T251111.

Thu, May 28, 12:48 PM · User-GoranSMilovanovic, WMDE-FUN-Sprint-2020-04-27, Wikidata - Reference Hunt - Sprint 6, Wikidata
Ladsgroup moved T251518: Integration tests for Pipe 2 from To do to Peer Review on the Wikidata - Reference Hunt - Sprint 6 board.

https://github.com/wmde/reference-island/pull/64

Thu, May 28, 10:13 AM · Wikidata - Reference Hunt - Sprint 6, User-Ladsgroup, Wikidata
Ladsgroup closed T253728: Fetch schema context is not working as Resolved.
Thu, May 28, 8:53 AM · User-Ladsgroup, Wikidata, Wikidata - Reference Treasure Hunt
Ladsgroup added a comment to T253728: Fetch schema context is not working.

I made this as a possible solution: https://github.com/wmde/reference-island/pull/63 Which seems easy and straightforward.

Thu, May 28, 8:42 AM · User-Ladsgroup, Wikidata, Wikidata - Reference Treasure Hunt
Ladsgroup claimed T251518: Integration tests for Pipe 2.
Thu, May 28, 8:19 AM · Wikidata - Reference Hunt - Sprint 6, User-Ladsgroup, Wikidata
Ladsgroup added a comment to T119173: RFC: Discourage use of MySQL's ENUM type.
  • Not all DBMSes support ENUM, for example sqlite that we officially support doesn't have enum datatype and the only way is to use "constraint" […]

We currently have ENUM in our core schema, e.g. categorylinks and image, afaik we don't have a separate schema for sqlite. So what do we do for these today? What are the caveats with/impact of that today?

Thu, May 28, 8:13 AM · Performance-Team (Radar), TechCom-RFC, MediaWiki-General, DBA
Ladsgroup added a comment to T111853: The href of be-tarask: interlanguage link points to the be-x-old domain.

Change 587863 had a related patch set uploaded (by Dvorapa; owner: Dvorapa):
[mediawiki/extensions/WikimediaMaintenance@master] [bugfix] Fix double redirect with be-tarask wiki

https://gerrit.wikimedia.org/r/587863

Thu, May 28, 7:54 AM · User-Ladsgroup, MW-1.35-notes (1.35.0-wmf.35; 2020-06-02), Wikimedia-Interwiki-links, WMF-deploy-2015-09-08_(1.26wmf22), WMF-deploy-2015-09-15_(1.26wmf23), WMF-deploy-2015-09-01_(1.26wmf21), Patch-For-Review, MediaWiki-extensions-WikimediaMaintenance, Wikimedia-Site-requests
Ladsgroup added a comment to T223410: Split group1 so that Commons and Wikidata aren't in the general group1, but their own buckets.

I'm slightly concerned by putting Wikidata in front of all production wikis. Issues in Wikidata usually propagates to all wikis, can Wikidata take it after group0?

Thu, May 28, 7:06 AM · Patch-For-Review, User-brennen, Release-Engineering-Team (Deployment services), Release-Engineering-Team-TODO, Wikidata, Commons
Ladsgroup added a comment to T246945: New Public Wiki for the API Portal.

hey @Ladsgroup thanks again for helping out with this. Does @CCicalese_WMF answer unblock us for this?

Thu, May 28, 7:02 AM · Patch-For-Review, Release-Engineering-Team-TODO, CPT Initiatives (API Gateway), User-brennen, User-Ladsgroup, Wiki-Setup (Create), User-Urbanecm, Core Platform Team

Wed, May 27

Ladsgroup added a comment to T238199: SpecialFewestRevisions::reallyDoQuery takes more than 9h to run.

With the announcement done, this can move forward on our side

Wed, May 27, 10:32 AM · Wikimedia-Incident, Wikimedia-database-error, MediaWiki-Special-pages, Wikidata
Ladsgroup created T253728: Fetch schema context is not working.
Wed, May 27, 8:36 AM · User-Ladsgroup, Wikidata, Wikidata - Reference Treasure Hunt

Tue, May 26

Ladsgroup updated the task description for T251863: Improve Storage Writing.
Tue, May 26, 4:22 PM · Wikidata - Reference Treasure Hunt, Wikidata
Ladsgroup added a comment to T253624: Evaluate the future of wikibase-codesniffer.

Note that phpcs ruleset is controllable per repository. The same way that we disable some rules in repositories not yet ready to follow a newer convention, it is also possible to enable additional rules on a per-repo bases.

Tue, May 26, 4:00 PM · MediaWiki-Codesniffer, Wikidata
Ladsgroup added a comment to T253624: Evaluate the future of wikibase-codesniffer.

I honestly think we should just drop wikibase codesniffer. We have been maintaining too much codebase.

Tue, May 26, 3:39 PM · MediaWiki-Codesniffer, Wikidata
Ladsgroup added a comment to T252091: RFC: Site-wide edit rate limiting with PoolCounter.

This proposal is effectively a dynamic rate limit except that instead of delivering an error message when it is exceeded, we will just hold the connection open, forcing the bot to wait. That's expensive in terms of server resources -- we'd rather have the client wait using only its own resources. A rate limit has a tunable parameter (the rate) which is not really knowable. Similarly, this proposal has a tunable parameter (the pool size) which is not really knowable. You have to tune the pool size down until the replag stops increasing, but then if the nature of the edits changes, or if the hardware changes, the optimal pool size will change.

I suggested at T202107 that the best method for globally controlling replication lag would be with a PID controller. A PID controller suppresses oscillation by having a memory of recent changes in the metric. The P (proportional) term is essentially as proposed at T240442 -- just back off proportionally as the lag increases. The problem with this is that it will settle into an equilibrium lag somewhere in the middle of the range. The I (integral) term addresses this by maintaining a rolling average and adjusting the control value until the average meets the desired value. This allows it to maintain approximately the same edit rate but with a lower average replication lag. The D (derivative) term causes the control value to be reduced more aggressively if the metric is rising quickly.

My proposal is to use a PID controller to set the Retry-After header. Clients would be strongly encouraged to respect that header. We could have say maxlag=auto to opt in to this system.

Tue, May 26, 2:03 PM · Sustainability (Incident Prevention), User-Addshore, Wikidata-Campsite, Wikidata, TechCom-RFC
Ladsgroup added a comment to T253364: Move Vue.js search development over to Gerrit.

May I request to keep the component library hosted in github? So the contribution path is easy, Project is discoverable and hosted in a familiar place for rest of the world to submit issues, fork and submit pull requests and so on?

Tue, May 26, 11:24 AM · Vue.js (Vue.js-Search)
Ladsgroup added a comment to T253245: Re-run SS1 to get an accurate list of whitelisted external identifiers.

https://github.com/wmde/reference-hunting-data/pull/4

Tue, May 26, 10:47 AM · User-Ladsgroup, Wikidata, Wikidata - Reference Treasure Hunt
Ladsgroup moved T251271: Improve Storage Reading from Doing to Peer Review on the Wikidata - Reference Hunt - Sprint 6 board.

https://github.com/wmde/reference-island/pull/60

Tue, May 26, 9:59 AM · User-Ladsgroup, Wikidata - Reference Hunt - Sprint 6, Wikidata
Ladsgroup added a comment to T251863: Improve Storage Writing.

https://github.com/wmde/reference-island/pull/60

Tue, May 26, 9:58 AM · Wikidata - Reference Treasure Hunt, Wikidata
Ladsgroup added a comment to T251863: Improve Storage Writing.

I sorta accidentally picked it up while working on the reading improvements. One thing that I realized is that even batching it to 2 item per write makes it quite faster:

amsa@amsa-Latitude-7480:~/workspace/ref2$ time python3 wikidatarefisland/run.py --step extract_items --side-service-input "whitelisted_ext_idefs.json" --input "/home/amsa/workspace/latest-all.json.gz" --output "extracted_unreferenced_statements.jsonl" --write-batch 0
Tue, May 26, 9:07 AM · Wikidata - Reference Treasure Hunt, Wikidata
Ladsgroup changed the visibility for P9611 (An Untitled Masterwork).
Tue, May 26, 9:01 AM · Wikidata, DBA
Ladsgroup closed T247834: Add introductory note to Special:ManageShortUrls as Resolved.
Tue, May 26, 7:06 AM · MW-1.35-notes (1.35.0-wmf.34; 2020-05-26), MediaWiki-extensions-UrlShortener
Ladsgroup claimed T251271: Improve Storage Reading.
Tue, May 26, 6:38 AM · User-Ladsgroup, Wikidata - Reference Hunt - Sprint 6, Wikidata

Mon, May 25

Ladsgroup placed T252787: Make Property ID visible in user language up for grabs.
Mon, May 25, 9:07 AM · User-Ladsgroup, Wikidata, Wikidata - Reference Treasure Hunt
Ladsgroup claimed T252787: Make Property ID visible in user language.
Mon, May 25, 8:20 AM · User-Ladsgroup, Wikidata, Wikidata - Reference Treasure Hunt
Ladsgroup claimed T253245: Re-run SS1 to get an accurate list of whitelisted external identifiers.
Mon, May 25, 6:46 AM · User-Ladsgroup, Wikidata, Wikidata - Reference Treasure Hunt

Sun, May 24

Ladsgroup added a comment to T231326: Explore ResourceLoader package files feature as way to reduce number of modules.

Thanks! A bunch of those involve CX1 which we would prefer not to touch at all until it is removed completely.

Sun, May 24, 6:19 AM · Performance-Team (Radar), ContentTranslation