Page MenuHomePhabricator

Deploy regular running of wikidata constraint checks using the job queue
Closed, ResolvedPublic3 Estimated Story Points

Description

Once T204022 is done and merged and deployed we can think about deploying the feature:

The feature should have a staged roll out, slowly ramping up the number of edits that the jobs are run for.
Throughout the time of the roll out metrics such as number of jobs in the queue etc should be monitored.
Throughout the rollout we also need to check the cache status and cache eviction rate, we probably can't fit results for all entities in the cache..
The job will only really take full effect when T204024 is also done persistently storing the data.

The config option was introduced in https://gerrit.wikimedia.org/r/#/c/463950/
The config options is wgWBQualityConstraintsEnableConstraintsCheckJobsRatio

Beta and test can probably be deployed quickly / in the same day (if everything goes fine).
wikidata.org should probably not proceed any faster than 1 increase per day.

Deployment List:

  • beta wikidata 100% - 16th Jan 2019
  • testwikidata 50% - 16th Jan 2019
  • testwikidata 100% - 16th Jan 2019
  • wikidata.org 1% - 16th Jan 2019
  • wikidata.org 5% - 17th Jan 2019
  • wikidata.org 10% - 22nd jan EU Morning 2019
  • wikidata.org 25% - 22nd jan EU Morning 2019
  • wikidata.org 40% - 1st Feb 2021
  • wikidata.org 50% - 11th Feb 2021
  • wikidata.org 60% - 12th Apr 2021
  • wikidata.org 70% - 3rd May 2021

...

  • wikidata.org 100%

When deploying and expecting a rate of around 10 jobs per second ping Services

Dashboards:

Details

ProjectBranchLines +/-Subject
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -0
mediawiki/services/change-propagation/jobqueue-deploymaster+2 -0
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+1 -0
mediawiki/extensions/WikibaseQualityConstraintswmf/1.33.0-wmf.13+11 -1
mediawiki/extensions/WikibaseQualityConstraintswmf/1.33.0-wmf.12+11 -1
mediawiki/extensions/WikibaseQualityConstraintsmaster+11 -1
operations/mediawiki-configmaster+0 -1
operations/mediawiki-configmaster+1 -0
operations/mediawiki-configmaster+1 -1
operations/mediawiki-configmaster+5 -0
operations/mediawiki-configmaster+4 -0
Show related patches Customize query in gerrit

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Looking at https://grafana.wikimedia.org/d/000000489/wikidata-query-service?orgId=1&var-cluster_name=wdqs-internal&from=now-90d&to=now im not seeing any throttled or banned requests for the internal cluster in the last 3 months.
Maybe the work on improving the updater etc means we might be able to bump this up further?

This month T240884: RFC: How to evaluate user-provided regular expressions was also discussed and we will try to implement a regex checking service so that we can complete T176312: Don’t check format constraint via SPARQL (safely evaluating user-provided regular expressions)

Will ping the query service team before we thinking about increasing this at all.

Change 484633 had a related patch set uploaded (by Jforrester; owner: Addshore):
[operations/mediawiki-config@master] wikidata: post edit constraint jobs on 50% of edits

https://gerrit.wikimedia.org/r/484633

Change 484635 had a related patch set uploaded (by Jforrester; owner: Addshore):
[operations/mediawiki-config@master] wikidata: post edit constraint jobs on 100% of edits

https://gerrit.wikimedia.org/r/484635

β€’ Addshore changed the task status from Stalled to Open.Feb 17 2020, 8:28 AM
β€’ Addshore lowered the priority of this task from Medium to Low.

Why is this priority low? From the product side I'd put this to at least medium due to it being a prerequisite for regular constraints runs and querying.

Change 484633 abandoned by Addshore:
wikidata: post edit constraint jobs on 50% of edits

https://gerrit.wikimedia.org/r/484633

Change 484635 abandoned by Addshore:
wikidata: post edit constraint jobs on 100% of edits

https://gerrit.wikimedia.org/r/484635

Per Adam the next step could be 40% for Wikidata.org, in case we want to be extra cautious

Change 660774 had a related patch set uploaded (by Rosalie Perside (WMDE); owner: Rosalie Perside (WMDE)):
[operations/mediawiki-config@master] wikidata: post edit constrain jobs on 50% of edits

https://gerrit.wikimedia.org/r/660774

Change 660774 merged by jenkins-bot:
[operations/mediawiki-config@master] wikidata: post edit constraint jobs on 40% of edits

https://gerrit.wikimedia.org/r/660774

Mentioned in SAL (#wikimedia-operations) [2021-02-01T12:19:03Z] <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:660774|wikidata: post edit constraint jobs on 40% of edits (T204031)]] (duration: 01m 03s)

According to JobQueue EventBus, the insertion rate of this job went up a bit yesterday, but it’s not too extreme (cursor is on the approximate timestamp of the latest deployment):

constraints-job-insertion-rate.png (334Γ—910 px, 39 KB)

I haven’t found any errors related to the job (constraintsRunCheck / CheckConstraintsJob) in Logstash, so everything seems to be fine so far.

Change 662967 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[operations/mediawiki-config@master] wikidata: post edit constraint jobs on 50% of edits

https://gerrit.wikimedia.org/r/662967

Change 662967 merged by jenkins-bot:
[operations/mediawiki-config@master] wikidata: post edit constraint jobs on 50% of edits

https://gerrit.wikimedia.org/r/662967

Mentioned in SAL (#wikimedia-operations) [2021-02-11T12:18:01Z] <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:662967|wikidata: post edit constraint jobs on 50% of edits (T204031)]] (up from 40%) (duration: 01m 08s)

Change 677928 had a related patch set uploaded (by Tonina Zhelyazkova; author: Tonina Zhelyazkova):

[operations/mediawiki-config@master] wikidata: post edit constraint jobs on 60% of edits

https://gerrit.wikimedia.org/r/677928

Restricted Application added a subscriber: Zabe. Β· View Herald TranscriptApr 8 2021, 2:59 PM

Change 677928 merged by jenkins-bot:

[operations/mediawiki-config@master] wikidata: post edit constraint jobs on 60% of edits

https://gerrit.wikimedia.org/r/677928

Mentioned in SAL (#wikimedia-operations) [2021-04-12T11:13:42Z] <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:677928|wikidata: post edit constraint jobs on 60% of edits (T204031)]] (duration: 01m 13s)

Looking good, I'll put it in waiting until camp feels ready for it to be back in TODO for the next jump

Change 682608 had a related patch set uploaded (by Tonina Zhelyazkova; author: Tonina Zhelyazkova):

[operations/mediawiki-config@master] wikidata: post edit constraint jobs on 70% of edits

https://gerrit.wikimedia.org/r/682608

Change 682608 merged by jenkins-bot:

[operations/mediawiki-config@master] wikidata: post edit constraint jobs on 70% of edits

https://gerrit.wikimedia.org/r/682608

Mentioned in SAL (#wikimedia-operations) [2021-05-03T11:04:41Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: f1a5ef0116c77b86b1abfb7bfa7d4ed363c69f61: wikidata: post edit constraint jobs on 70% of edits (T204031) (duration: 00m 57s)

This comment was removed by Michael.

Should we wait until T176312: Don’t check format constraint via SPARQL (safely evaluating user-provided regular expressions) is done and then move forward with it. It looks sensible to do it this way.

We can do (as it is being tackled right now)

Ladsgroup changed the task status from Stalled to Open.Aug 9 2021, 8:23 AM
Ladsgroup claimed this task.

Change 710925 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Increase post edit constraint jobs to 85% of edits

https://gerrit.wikimedia.org/r/710925

Change 710925 merged by jenkins-bot:

[operations/mediawiki-config@master] Increase post edit constraint jobs to 85% of edits

https://gerrit.wikimedia.org/r/710925

Mentioned in SAL (#wikimedia-operations) [2021-08-09T09:49:30Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:710925|Increase post edit constraint jobs to 85% of edits (T204031)]] (duration: 00m 58s)

Change 710936 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Enable post edit constraint jobs in all edits

https://gerrit.wikimedia.org/r/710936

Change 710936 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable post edit constraint jobs in all edits

https://gerrit.wikimedia.org/r/710936

Mentioned in SAL (#wikimedia-operations) [2021-08-09T10:27:05Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:710936|Enable post edit constraint jobs in all edits (T204031)]] (duration: 00m 58s)