Page MenuHomePhabricator
Create Task
Maniphest T249196

Test the impact of the wdqs updater performance by disabling values cleanup
Closed, ResolvedPublic
Actions

Description

Lag on the updater is rising:

wdqs_lag.png (780×1 px, 146 KB)

References cleanup has already been disabled to investigate another issue without noticeable negative impact, we should try to disable the values cleanup to see if it helps with catching up the lag.

Details

ProjectBranchLines +/-Subject
wikidata/query/rdfmaster+16 -4Disable values cleanup
Customize query in gerrit

Event Timeline

• dcausse created this task.Apr 2 2020, 8:46 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 2 2020, 8:46 AM
gerritbot added a comment.Apr 2 2020, 9:08 AM

Change 585443 had a related patch set uploaded (by DCausse; owner: DCausse):
[wikidata/query/rdf@master] Disable values cleanup

https://gerrit.wikimedia.org/r/585443

gerritbot added a project: Patch-For-Review.Apr 2 2020, 9:08 AM
• dcausse updated the task description. (Show Details)Apr 2 2020, 9:13 AM
gerritbot added a comment.Apr 2 2020, 9:22 AM

Change 585443 merged by jenkins-bot:
[wikidata/query/rdf@master] Disable values cleanup

https://gerrit.wikimedia.org/r/585443

Maintenance_bot removed a project: Patch-For-Review.Apr 2 2020, 10:10 AM
Stashbot added a comment.Apr 3 2020, 12:44 PM

Mentioned in SAL (#wikimedia-operations) [2020-04-03T12:44:58Z] <dcausse@deploy1001> Started deploy [wdqs/wdqs@23495ae]: deploying wdqs 0.3.17 to wdqs1007: testing T249196

Stashbot added a comment.Apr 3 2020, 12:45 PM

Mentioned in SAL (#wikimedia-operations) [2020-04-03T12:45:41Z] <dcausse@deploy1001> Finished deploy [wdqs/wdqs@23495ae]: deploying wdqs 0.3.17 to wdqs1007: testing T249196 (duration: 00m 43s)

Stashbot added a comment.Apr 7 2020, 7:40 AM

Mentioned in SAL (#wikimedia-operations) [2020-04-07T07:40:16Z] <dcausse@deploy1001> Started deploy [wdqs/wdqs@23495ae]: deploying wdqs 0.3.17 to wdqs2002: T249196

Stashbot added a comment.Apr 7 2020, 7:41 AM

Mentioned in SAL (#wikimedia-operations) [2020-04-07T07:41:44Z] <dcausse@deploy1001> Finished deploy [wdqs/wdqs@23495ae]: deploying wdqs 0.3.17 to wdqs2002: T249196 (duration: 01m 28s)

• dcausse edited projects, added Wikidata-Query-Service; removed WDQS-Optimizer.Apr 7 2020, 7:42 AM
Restricted Application added a project: Wikidata. · View Herald TranscriptApr 7 2020, 7:42 AM
• dcausse claimed this task.Apr 7 2020, 5:00 PM
• dcausse moved this task from Incoming to To Be Deployed on the Discovery-Search (Current work) board.EditedApr 7 2020, 6:39 PM

The lag on wdqs1007 has been absorbed much faster than other eqiad nodes.

lag_wdqs1007.png (202×888 px, 53 KB)

Comparing codfw nodes the lag is a bit better on a node where the cleanup has been removed (wdqs2002 removed vs wdqs2003 cleanup enabled) but the impact is not particularly impressive, wdqs2002 lag still climbs relying on back pressure propagation through maxLag (sawtooth shape)

lag_wdqs2003.png (169×852 px, 54 KB)

Overall I think that this cleanup does not bring much to the system, while it was being enabled for more than 3 days I didn't see a dramatic climb in the number of triples.
I think opting for more frequent reloads is much more efficient approach.

Gehel moved this task from Incoming to Scaling on the Wikidata-Query-Service board.Apr 8 2020, 12:33 PM
• dcausse moved this task from To Be Deployed to Needs Reporting on the Discovery-Search (Current work) board.Apr 20 2020, 5:30 PM
TJones closed this task as Resolved.Apr 29 2020, 2:39 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL