Ladsgroup (Amir Sarabadani)Administrator
Master of introducing new bugs

Projects (28)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Oct 6 2014, 9:53 PM (214 w, 6 d)
Roles
Administrator
Availability
Available
IRC Nick
Amir1
LDAP User
Ladsgroup
MediaWiki User
Ladsgroup [ Global Accounts ]

Recent Activity

Sat, Nov 17

Ladsgroup awarded Blog Post: Bring in 'da noise, bring in defunct. It's a zombie party! a Love token.
Sat, Nov 17, 10:38 PM · Continuous-Integration-Infrastructure, Release-Engineering-Team

Fri, Nov 16

GitHub <noreply@github.com> committed rORESf1e463e676c6: Merge 0fcfc3a5e8c63065b1c99173e1067c8bbca6bdcf into… (authored by Ladsgroup).
Merge 0fcfc3a5e8c63065b1c99173e1067c8bbca6bdcf into…
Fri, Nov 16, 6:40 PM
Ladsgroup committed rORES0fcfc3a5e8c6: Remove all of celery configs (authored by Ladsgroup).
Remove all of celery configs
Fri, Nov 16, 6:40 PM
Ladsgroup committed rORES35fc735f06c9: Remove all of celery configs (authored by Ladsgroup).
Remove all of celery configs
Fri, Nov 16, 6:25 PM

Thu, Nov 15

Ladsgroup claimed T170950: ORES deployment finish "successfully" even when uwsgi and celery fail to successfully start up.
Thu, Nov 15, 10:13 PM · User-Ladsgroup, Scoring-platform-team (Current), Scap, ORES
Ladsgroup added a comment to T207560: Carry out the 2018 fawiki elections on votewiki.

It's varnish: https://vote.wikimedia.org/wiki/Main_Page?ll

Thu, Nov 15, 6:40 PM · Patch-For-Review, Trust-and-Safety, Wikimedia-Site-requests
Ladsgroup added a comment to T207560: Carry out the 2018 fawiki elections on votewiki.

This is weird. It's pretty plainly still in Persian so not sure what happened with this patch?

Thu, Nov 15, 6:39 PM · Patch-For-Review, Trust-and-Safety, Wikimedia-Site-requests
Ladsgroup moved T209459: Wikidata recentchanges API request with ORESscores parameter returns 0 results from Review to Pending deployment on the Scoring-platform-team (Current) board.
Thu, Nov 15, 6:29 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), Performance, ORES, MediaWiki-Recent-changes, Growth-Team, Wikidata
Ladsgroup moved T208623: ORES should log 500 responses from Review to Done on the Scoring-platform-team (Current) board.
Thu, Nov 15, 6:29 PM · User-Ladsgroup, Scoring-platform-team (Current), ORES
Ladsgroup claimed T209587: Migrate ores celery configs to celery 4.
Thu, Nov 15, 6:28 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), ORES
Ladsgroup closed T181559: Investigate redis-cluster or other techniques for making Redis not a single point of failure. as Resolved.
Thu, Nov 15, 6:26 PM · User-Ladsgroup, Scoring-platform-team (Current), Wikimedia-Incident, Operations
Ladsgroup closed T181559: Investigate redis-cluster or other techniques for making Redis not a single point of failure., a subtask of T181538: ORES overload incident, 2017-11-28, as Resolved.
Thu, Nov 15, 6:26 PM · Wikimedia-Incident, Operations, Scoring-platform-team
Ladsgroup claimed T181559: Investigate redis-cluster or other techniques for making Redis not a single point of failure..

We decided to go with redis-sentinel

Thu, Nov 15, 6:26 PM · User-Ladsgroup, Scoring-platform-team (Current), Wikimedia-Incident, Operations
Ladsgroup closed T181632: Celery manager implodes horribly if Redis goes down as Resolved.
Thu, Nov 15, 6:25 PM · User-Ladsgroup, Scoring-platform-team (Current), Wikimedia-Incident, Operations
Ladsgroup closed T181632: Celery manager implodes horribly if Redis goes down, a subtask of T181538: ORES overload incident, 2017-11-28, as Resolved.
Thu, Nov 15, 6:25 PM · Wikimedia-Incident, Operations, Scoring-platform-team
Ladsgroup raised the priority of T122676: Implement sentinel for ORES production Redis from Normal to High.
Thu, Nov 15, 6:25 PM · Scoring-platform-team, ORES
Ladsgroup renamed T122676: Implement sentinel for ORES production Redis from Implement twemproxy for ORES production Redis to Implement sentinel for ORES production Redis.
Thu, Nov 15, 6:23 PM · Scoring-platform-team, ORES
Ladsgroup moved T196889: Investigate what is creating Redis transactions and whether it can be fixed from Active to Done on the Scoring-platform-team (Current) board.

This is done, we are not fixing transcations. We'll work on sentinel

Thu, Nov 15, 6:22 PM · User-Ladsgroup, Scoring-platform-team (Current), ORES
Ladsgroup closed T196889: Investigate what is creating Redis transactions and whether it can be fixed, a subtask of T122676: Implement sentinel for ORES production Redis, as Resolved.
Thu, Nov 15, 6:21 PM · Scoring-platform-team, ORES
Ladsgroup closed T196889: Investigate what is creating Redis transactions and whether it can be fixed as Resolved.
Thu, Nov 15, 6:21 PM · User-Ladsgroup, Scoring-platform-team (Current), ORES
Ladsgroup awarded T206341: Evaluate scalability and performance of PHP7 compared to HHVM a Like token.
Thu, Nov 15, 6:19 PM · Performance-Team, Operations
Ladsgroup committed rORESDEPLOYdba11e964064: Bump ores to HEAD (authored by Ladsgroup).
Bump ores to HEAD
Thu, Nov 15, 6:15 PM
GitHub <noreply@github.com> committed rORES2e0476b67d70: Merge 125dbe979f0e3dceda78a495c850cff790d39e1d into… (authored by Ladsgroup).
Merge 125dbe979f0e3dceda78a495c850cff790d39e1d into…
Thu, Nov 15, 6:05 PM
Ladsgroup committed rORESDEPLOY51cdf6b28dde: Bump ores to HEAD (authored by Ladsgroup).
Bump ores to HEAD
Thu, Nov 15, 5:22 PM
Ladsgroup reopened T208687: Convert SecurePoll to use OOUI as "Open".

I only did the jump form, there are lots of forms in this extension that would benefit from migrating to OOUI. Specially the ballot forms.

Thu, Nov 15, 4:05 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), User-Ladsgroup, MediaWiki-extensions-SecurePoll, UI-Standardization
Ladsgroup reopened T208687: Convert SecurePoll to use OOUI, a subtask of T122014: Convert all extensions/skins to OOUI, as Open.
Thu, Nov 15, 4:05 PM · Goal, UI-Standardization
Ladsgroup moved T209459: Wikidata recentchanges API request with ORESscores parameter returns 0 results from Active to Review on the Scoring-platform-team (Current) board.
Thu, Nov 15, 3:22 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), Performance, ORES, MediaWiki-Recent-changes, Growth-Team, Wikidata
Ladsgroup created T209587: Migrate ores celery configs to celery 4.
Thu, Nov 15, 2:16 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), ORES
Ladsgroup added a comment to T122676: Implement sentinel for ORES production Redis.

I looked at sentinel. It's a little bit complex but easily doable. We probably need to do install redis on ores nodes and make them to run sentinel (Example 3 in https://redis.io/topics/sentinel) Setting up the configuration would be a little bit hard and basically all needs to be done via puppet. One other concern is dockerizing ores. That's going to be even harder with docker.

Thu, Nov 15, 2:07 PM · Scoring-platform-team, ORES
Ladsgroup added a comment to T122676: Implement sentinel for ORES production Redis.
Thu, Nov 15, 11:40 AM · Scoring-platform-team, ORES
Ladsgroup merged Restricted Task into T207297: Phan SecurityCheck-XSS and SecurityCheck-SQLInjection errors in SecurePoll extension.
Thu, Nov 15, 11:26 AM · Patch-For-Review, MediaWiki-extensions-SecurePoll, Security
Ladsgroup triaged T205972: Fixup Phan errors in SecurePoll as High priority.

Still errors and I can't merge anything.

Thu, Nov 15, 11:25 AM · MW-1.33-notes (1.33.0-wmf.4; 2018-11-13), Patch-For-Review, phan-taint-check-plugin, MediaWiki-extensions-SecurePoll

Wed, Nov 14

Ladsgroup updated the task description for T206325: Use wikibase/data-model 9.0 everywhere.
Wed, Nov 14, 9:42 PM · Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MW-1.33-notes (1.33.0-wmf.4; 2018-11-13), MediaWiki-Vendor, MW-1.32-notes (WMF-deploy-2018-10-16 (1.32.0-wmf.26)), Patch-For-Review, wikidata-tech-focus, Wikibase-DataModel, Wikidata
Ladsgroup moved T209370: Make wikibase extensions compatible with both version 7.0 & 9.0 from Peer Review to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Wed, Nov 14, 9:42 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-Vendor, wikidata-tech-focus, Wikibase-DataModel, Wikidata
Ladsgroup removed a project from T209370: Make wikibase extensions compatible with both version 7.0 & 9.0: Patch-For-Review.
Wed, Nov 14, 9:42 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), MediaWiki-Vendor, wikidata-tech-focus, Wikibase-DataModel, Wikidata
Ladsgroup moved T194163: Drop change_tag.ct_tag column from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Wed, Nov 14, 8:20 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, MediaWiki-Database, MediaWiki-Change-tagging
Ladsgroup added a comment to T209525: Migrate tag_summary usage to change_tag and drop the table.

Should I turn this into an RFC?

Wed, Nov 14, 7:23 PM · Patch-For-Review, MediaWiki-Change-tagging
Ladsgroup created T209525: Migrate tag_summary usage to change_tag and drop the table.
Wed, Nov 14, 7:22 PM · Patch-For-Review, MediaWiki-Change-tagging
Ladsgroup closed T185847: Fatal exception DBQueryError "Lock wait timeout exceeded" on Special:Tags as Resolved.

The backend for Special:Tags have been changed a while ago. This should not happen anymore. Feel free to reopen it if you encounter such things.

Wed, Nov 14, 7:17 PM · User-Ladsgroup, MediaWiki-Special-pages, MediaWiki-Database, Wikimedia-production-error
Ladsgroup closed T185847: Fatal exception DBQueryError "Lock wait timeout exceeded" on Special:Tags, a subtask of T30499: 1205: Lock wait timeout exceeded; try restarting transaction (tracking), as Resolved.
Wed, Nov 14, 7:17 PM · Technical-Debt, Tracking, MediaWiki-Database
Ladsgroup moved T194163: Drop change_tag.ct_tag column from To Do to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Wed, Nov 14, 6:27 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, MediaWiki-Database, MediaWiki-Change-tagging
Ladsgroup claimed T194163: Drop change_tag.ct_tag column.
Wed, Nov 14, 6:27 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, MediaWiki-Database, MediaWiki-Change-tagging
Ladsgroup moved T200064: Update documentation on mediawiki.org: Manual:Change_tag_table from To Do to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

I did it today.

Wed, Nov 14, 6:23 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Documentation, MediaWiki-Change-tagging
Ladsgroup claimed T200064: Update documentation on mediawiki.org: Manual:Change_tag_table.
Wed, Nov 14, 6:23 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Documentation, MediaWiki-Change-tagging
Ladsgroup added a comment to T209508: PHP 7.2 gate-and-submit for some extensions fails due to deprecated function each() in CirrusSearch / Elastica.

Property Suggester is also failing with this: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/PropertySuggester/+/473545

Wed, Nov 14, 5:49 PM · PHP 7.2 support, MediaWiki-extensions-PropertySuggester, Continuous-Integration-Config, Discovery-Search, CirrusSearch, Wikibase-Quality-Constraints, Wikibase-Quality, Wikidata
Ladsgroup moved T209207: Enable arbitrary access on Incubator from Doing to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Wed, Nov 14, 5:27 PM · User-Ladsgroup, Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Ladsgroup added a comment to T186006: References aren't readable to unprivileged users on Wikidata when an item is protected.

I put up this workaround for now: https://www.wikidata.org/w/index.php?title=MediaWiki:Gadget-ProtectionIndicators.js&diff=789802852&oldid=393479817
I'm looking for a proper fix

Wed, Nov 14, 4:02 PM · Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Need-volunteer, Regression, MediaWiki-extensions-WikibaseRepository, Wikidata
Ladsgroup updated the task description for T209207: Enable arbitrary access on Incubator.
Wed, Nov 14, 3:22 PM · User-Ladsgroup, Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Ladsgroup claimed T209207: Enable arbitrary access on Incubator.
Wed, Nov 14, 3:19 PM · User-Ladsgroup, Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Wikidata
Ladsgroup moved T208846: Start reading from change_tag_def on wikidatawiki from To Do to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.
Wed, Nov 14, 2:21 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, wikidata-tech-focus, MediaWiki-Change-tagging, Wikidata
Ladsgroup moved T202543: Enable arbitrary access on oldwikisource & document the process from To Do to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board.

Done: https://wikitech.wikimedia.org/wiki/WMDE/Wikidata/Deployment

Wed, Nov 14, 2:13 PM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), wikidata-tech-focus, Patch-For-Review, Wikimedia-Site-requests, Wikidata
Ladsgroup added a comment to T205016: The license should be customizable, currently, CC0 license is hardcoded into the .ttl output.

Do we have a good place to document this? Because it seems like one of those things that people otherwise overlook easily and since it isn't configured for Wikidata they also can't see it simply by copying from our config.

Wed, Nov 14, 1:01 PM · MW-1.33-notes (1.33.0-wmf.4; 2018-11-13), Patch-For-Review, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, Wikidata, Federated-Wikibase-Workshops@NewYork-2018
Ladsgroup claimed T209459: Wikidata recentchanges API request with ORESscores parameter returns 0 results.

I will work on it.

Wed, Nov 14, 12:58 PM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), Performance, ORES, MediaWiki-Recent-changes, Growth-Team, Wikidata
Ladsgroup changed the status of T208846: Start reading from change_tag_def on wikidatawiki, a subtask of T185355: Normalize change tag schema, from Stalled to Open.
Wed, Nov 14, 10:25 AM · MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), Wikidata, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), User-Ladsgroup, MW-1.32-notes (WMF-deploy-2018-05-15 (1.32.0-wmf.4)), Wikidata-Ministry-Of-Magic-Tech-Debt, Wikidata-Ministry-Of-Magic, TechCom-RFC (TechCom-Approved), MediaWiki-Database, MediaWiki-Change-tagging, Patch-For-Review
Ladsgroup changed the status of T208846: Start reading from change_tag_def on wikidatawiki from Stalled to Open.
Wed, Nov 14, 10:25 AM · User-Ladsgroup, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞), Patch-For-Review, wikidata-tech-focus, MediaWiki-Change-tagging, Wikidata

Tue, Nov 13

RandomDSdevel awarded T209036: The title "Foo" does not refer to an existing page a Grey Medal token.
Tue, Nov 13, 2:35 AM · MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), Patch-For-Review, TCB-Team, Two-Column-Edit-Conflict-Merge

Fri, Nov 9

Ladsgroup added a comment to T191964: Clickstream dataset for Persian Wikipedia only includes external values.

^ I made this patch but I basically had no way to test things. Please double check and if possible run it for a short period of time before merging.

Fri, Nov 9, 6:35 PM · Patch-For-Review, Analytics-Kanban, Analytics
Ladsgroup added a comment to T182249: Diagnose and fix 4.5k req/min ceiling for ores* requests.

We can't increase number of our celery workers due to memory allocation. Maybe we need to tackle T182350: Profile ORES code memory use, optimize pre-fork allocation then fix things and then come back to increasing number of celery workers. I would encourage against lifting the connection limit but making it a little bit higher would be nice. Also, providing all of ores scores in the analytics data lake, or the brand new hadoop/presto cluster for labs (that is being built), or in dumps.wikimedia.org would lift most of the need for the researchers.

Fri, Nov 9, 5:24 PM · Scoring-platform-team, Operations, Performance, ORES
Ladsgroup added a comment to T182222: Create Grafana graph to show number of ORES API requests per user-agent.

Given that we have the 4 parallel connection per IP in place and difficulties of implementing this. I propose we decline this ticket. We need some better protections against malicious DDoS though but I doubt this would help.

Fri, Nov 9, 4:34 PM · Wikimedia-Incident, ORES, monitoring, Scoring-platform-team
Ladsgroup added a comment to T191842: Deployment git server can't supply ORES hosts in parallel.

With the new number of 9 parallel connection and 14 minutes to deploy (down from around half an hour), I think it's good now.

Fri, Nov 9, 3:09 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), Scap, Operations, Release-Engineering-Team
Ladsgroup moved T191842: Deployment git server can't supply ORES hosts in parallel from Pending deployment to Done on the Scoring-platform-team (Current) board.
Fri, Nov 9, 3:06 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), Scap, Operations, Release-Engineering-Team
Ladsgroup moved T209060: ores returns error for draftquality models from Pending deployment to Done on the Scoring-platform-team (Current) board.
Fri, Nov 9, 3:06 PM · Patch-For-Review, Scoring-platform-team (Current), User-Ladsgroup, ORES
Gerrit Code Review <gerrit@wikimedia.org> committed rORESDEPLOYbb39f4b3d6e9: Merge "Increase number of parallel connections to 9" (authored by Ladsgroup).
Merge "Increase number of parallel connections to 9"
Fri, Nov 9, 2:52 PM
Ladsgroup committed rORESWHEELS51b4e49827e1: Untrack non-wheel files from git lfs (authored by Ladsgroup).
Untrack non-wheel files from git lfs
Fri, Nov 9, 2:11 PM
Ladsgroup moved T209060: ores returns error for draftquality models from Active to Pending deployment on the Scoring-platform-team (Current) board.
Fri, Nov 9, 1:59 PM · Patch-For-Review, Scoring-platform-team (Current), User-Ladsgroup, ORES
Ladsgroup added a comment to T209060: ores returns error for draftquality models.

Deployed the change on beta, it works. Will deploy it later next week.

Fri, Nov 9, 1:58 PM · Patch-For-Review, Scoring-platform-team (Current), User-Ladsgroup, ORES
Ladsgroup committed rORESDEPLOY072880573109: Bump wheels to HEAD (authored by Ladsgroup).
Bump wheels to HEAD
Fri, Nov 9, 1:36 PM

Thu, Nov 8

Ladsgroup added a comment to T199133: Find top 15 target projects that could use Selenium tests to prevent incidents.

Just saying, the "ORES" service and backend of the ORES extension is being maintained by scoring platform team not growth.

Thu, Nov 8, 6:19 PM · Release-Engineering-Team (Kanban), User-zeljkofilipin
Ladsgroup added a comment to T196889: Investigate what is creating Redis transactions and whether it can be fixed.

Running redis-cli monitor on deployment-ores01 gives these kind of transactions:

Thu, Nov 8, 5:19 PM · User-Ladsgroup, Scoring-platform-team (Current), ORES
Ladsgroup added a comment to T122676: Implement sentinel for ORES production Redis.

Which now brings us to the question of what's the next step? Should we reinvent the wheel and make a basic nutcracker that is able to handle transactions? Should we change to use RabbitMQ for the broker and redis for other stuff? (In that case we can put the redis behind the nutcracker and I have no idea how to handle RabbitMQ). Fork celery and drop the transaction part? (Nooo)

Thu, Nov 8, 5:10 PM · Scoring-platform-team, ORES
Ladsgroup added a comment to T122676: Implement sentinel for ORES production Redis.

codfw has been migrated to use nutcracker and reverted. This has backfired majestically. The reason being

[2017-07-06 13:59:15.068] nc_redis.c:1092 parsed unsupported command 'MULTI'

Per https://github.com/twitter/twemproxy/blob/master/notes/redis.md#transactions this is not supported. We need to figure out why that command is being used.

Thu, Nov 8, 4:21 PM · Scoring-platform-team, ORES
Ladsgroup lowered the priority of T209060: ores returns error for draftquality models from Unbreak Now! to High.

With the hotfix, it's back to normal but we can't deploy anything because it might break them again.

Thu, Nov 8, 3:30 PM · Patch-For-Review, Scoring-platform-team (Current), User-Ladsgroup, ORES
Ladsgroup triaged T209060: ores returns error for draftquality models as Unbreak Now! priority.

Apparently in scap deploy, git lfs pull didn't work on all of wheels assets (only on the *.whl files and not on the corpora text files):

ladsgroup@deployment-ores01:/srv/deployment/ores/deploy/submodules/wheels$ sudo git lfs pull
Git LFS: (55 of 55 files) 96.63 KB / 96.63 KB

That is broken on production now.

Thu, Nov 8, 3:09 PM · Patch-For-Review, Scoring-platform-team (Current), User-Ladsgroup, ORES
Ladsgroup added a comment to T209060: ores returns error for draftquality models.

I can download it on my laptop but my connection is super slow but it's not downloading anything on labs nodes or prod stat machines :/

Thu, Nov 8, 2:58 PM · Patch-For-Review, Scoring-platform-team (Current), User-Ladsgroup, ORES
Ladsgroup updated subscribers of T209060: ores returns error for draftquality models.

Git lfs is broken:

ladsgroup@stat1007:~$ git clone ssh://ladsgroup@gerrit.wikimedia.org:29418/research/ores/wheels
.
.
.
ladsgroup@stat1007:~/wheels$ git lfs pull
Git LFS: (0 of 203 files) 0 B / 212.78 MB                                      EGit LFS: (0 of 203 files) 0 B / 212.78 MB

Doesn't download anything. @thcipriani @mmodell

Thu, Nov 8, 2:55 PM · Patch-For-Review, Scoring-platform-team (Current), User-Ladsgroup, ORES
Ladsgroup added a comment to T209060: ores returns error for draftquality models.

I have the feeling that rebuilding the wheels with git lfs caused this (that was the only thing that was deployed yesterday)

Thu, Nov 8, 2:33 PM · Patch-For-Review, Scoring-platform-team (Current), User-Ladsgroup, ORES
Ladsgroup created T209060: ores returns error for draftquality models.
Thu, Nov 8, 2:33 PM · Patch-For-Review, Scoring-platform-team (Current), User-Ladsgroup, ORES
Ladsgroup closed T172629: Train/test wp10 model for fawiki as Resolved.

This is done

Thu, Nov 8, 1:50 PM · User-Ladsgroup, ORES, Scoring-platform-team
Ladsgroup moved T191842: Deployment git server can't supply ORES hosts in parallel from Active to Pending deployment on the Scoring-platform-team (Current) board.
Thu, Nov 8, 12:00 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), Scap, Operations, Release-Engineering-Team
Ladsgroup committed rORESDEPLOY6c722fb30ca6: Increase number of parallel connections to 9 (authored by Ladsgroup).
Increase number of parallel connections to 9
Thu, Nov 8, 11:14 AM
Ladsgroup added a project to T209036: The title "Foo" does not refer to an existing page: Two-Column-Edit-Conflict-Merge.
Thu, Nov 8, 10:47 AM · MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), Patch-For-Review, TCB-Team, Two-Column-Edit-Conflict-Merge
Ladsgroup triaged T209036: The title "Foo" does not refer to an existing page as Unbreak Now! priority.
Thu, Nov 8, 10:47 AM · MW-1.33-notes (1.33.0-wmf.3; 2018-11-06), Patch-For-Review, TCB-Team, Two-Column-Edit-Conflict-Merge

Wed, Nov 7

Ladsgroup moved T191842: Deployment git server can't supply ORES hosts in parallel from Pending deployment to Active on the Scoring-platform-team (Current) board.

Now deployment time has been reduced from 22 minutes to 17 minutes. I will increase the number of parallel connections from 5 to 8 in the next try.

Wed, Nov 7, 9:23 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), Scap, Operations, Release-Engineering-Team
Ladsgroup moved T197096: [Epic] Use LFS for large ORES files from Pending deployment to Done on the Scoring-platform-team (Current) board.
Wed, Nov 7, 9:21 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current)
Ladsgroup moved T204724: Extend ORES graph support to all langauges that have WP10 ORES model from Untriaged to Monitor on the Scoring-platform-team board.
Wed, Nov 7, 8:38 PM · Scoring-platform-team, Education-Program-Dashboard
Ladsgroup moved T191842: Deployment git server can't supply ORES hosts in parallel from Active to Pending deployment on the Scoring-platform-team (Current) board.
Wed, Nov 7, 8:35 PM · Patch-For-Review, User-Ladsgroup, Scoring-platform-team (Current), Scap, Operations, Release-Engineering-Team
Ladsgroup committed rORESWHEELS01cb1c57c313: Rebuild wheels for new flask and celery 4 Bug: T208258 Bug: T178441 Change-Id… (authored by Ladsgroup).
Rebuild wheels for new flask and celery 4 Bug: T208258 Bug: T178441 Change-Id…
Wed, Nov 7, 7:20 PM
Gerrit Code Review <gerrit@wikimedia.org> committed rORESWHEELS38c4c202024e: Merge "Update revscoring wheel to 2.0.10" (authored by Ladsgroup).
Merge "Update revscoring wheel to 2.0.10"
Wed, Nov 7, 7:20 PM
Ladsgroup committed rORESWHEELS9b54d272a118: Add .gitreview Change-Id: Ic9b947adf5b1c7d7c3a1aee80ac97de6d213fb10 (authored by Ladsgroup).
Add .gitreview Change-Id: Ic9b947adf5b1c7d7c3a1aee80ac97de6d213fb10
Wed, Nov 7, 7:20 PM
Ladsgroup committed rORESWHEELSa76986588a6f: Delete unneeded wheels Change-Id: Id8326c6a7cf75d5c7916dba342ab5527e4d309c8 (authored by Ladsgroup).
Delete unneeded wheels Change-Id: Id8326c6a7cf75d5c7916dba342ab5527e4d309c8
Wed, Nov 7, 7:20 PM
Ladsgroup committed rORESWHEELS4ea55552f6da: Bumps revscoring to 1.3.6 Change-Id: I7ddb51e548c110a60972bc282aca2c5426ce8fd4 (authored by Halfak).
Bumps revscoring to 1.3.6 Change-Id: I7ddb51e548c110a60972bc282aca2c5426ce8fd4
Wed, Nov 7, 7:20 PM
Ladsgroup committed rORESWHEELS0d11df25ec5f: Bumps revscoring to 1.3.4 Change-Id: I302c30455e582f504eb01e69a5a3c93dabc53b16 (authored by Halfak).
Bumps revscoring to 1.3.4 Change-Id: I302c30455e582f504eb01e69a5a3c93dabc53b16
Wed, Nov 7, 7:20 PM
Ladsgroup committed rORESWHEELS414246a2cab3: Bumps yamlconf version Change-Id: Ib3dc249fce014b99ac5c7c2064974cd5143e2d12 (authored by Halfak).
Bumps yamlconf version Change-Id: Ib3dc249fce014b99ac5c7c2064974cd5143e2d12
Wed, Nov 7, 7:20 PM
Ladsgroup committed rORESWHEELSb2a2613fdbad: Adds mmh3 and mysqltsv. Bumps yamlconf. Change-Id… (authored by Halfak).
Adds mmh3 and mysqltsv. Bumps yamlconf. Change-Id…
Wed, Nov 7, 7:20 PM
Ladsgroup committed rORESWHEELSf33e317f2233: Removes deltas & mwapi duplicates. Change-Id… (authored by Halfak).
Removes deltas & mwapi duplicates. Change-Id…
Wed, Nov 7, 7:20 PM
Gerrit Code Review <gerrit@wikimedia.org> committed rORESWHEELS70a864d1cbbe: Merge "Bumps revscoring to 1.3.3, pywikibase to 0.0.3, deltas to 0.4.2, and… (authored by Ladsgroup).
Merge "Bumps revscoring to 1.3.3, pywikibase to 0.0.3, deltas to 0.4.2, and…
Wed, Nov 7, 7:20 PM
Gerrit Code Review <gerrit@wikimedia.org> committed rORESWHEELSdb0476e2af4b: Merge "Revert "Revert "Bumps revscoring to 1.3.1""" (authored by Ladsgroup).
Merge "Revert "Revert "Bumps revscoring to 1.3.1"""
Wed, Nov 7, 7:20 PM
Ladsgroup added a reverting change for rORESWHEELS25157a3f5dd7: Revert "Bumps revscoring to 1.3.1": rORESWHEELS43137edba5ca: Revert "Revert "Bumps revscoring to 1.3.1"" This reverts commit….
Wed, Nov 7, 7:20 PM
Ladsgroup committed rORESWHEELS43137edba5ca: Revert "Revert "Bumps revscoring to 1.3.1"" This reverts commit… (authored by Ladsgroup).
Revert "Revert "Bumps revscoring to 1.3.1"" This reverts commit…
Wed, Nov 7, 7:20 PM
Gerrit Code Review <gerrit@wikimedia.org> committed rORESWHEELS29b05a3d4682: Merge "Revert "Revert "Bumps pyenchant to 1.6.8""" (authored by Ladsgroup).
Merge "Revert "Revert "Bumps pyenchant to 1.6.8"""
Wed, Nov 7, 7:20 PM