Page MenuHomePhabricator
Create Task
Maniphest T291658

updateMenteeData.php submits a lot of DB queries
Closed, ResolvedPublic
Actions

Description

In T277862#7374919, @Ladsgroup wrote:

Took the binlog of the primary db and ordered them by the number of invocation (it's not a great metric but better than nothing):

MediaWiki\Extension\OATHAuth\Hook\LoadExtensionSchemaUpdates\UpdateTables::switchTOTPScratchTokensToArray www-data@deploy... 14784
GrowthExperiments\Maintenance\UpdateMenteeData::execute www-data@deploy... 6933
Wikimedia\DependencyStore\SqlModuleDependencyStore::storeMulti 5733
DatabaseUpdater::purgeCache www-data@deploy... 2160
User::saveSettings 2032

We don't need to run that many write queries. @Ladsgroup suggested to ditch upsert, manually check for row existance/current data, and do nothing/insert/update as needed.

Details

SubjectRepoBranchLines +/-
updateMenteeData.php: Decrease number of write queries mademediawiki/extensions/GrowthExperimentswmf/1.38.0-wmf.1+26 -13
updateMenteeData.php: Decrease number of write queries mademediawiki/extensions/GrowthExperimentsmaster+26 -13
Customize query in gerrit

Related Objects
Search...

Event Timeline

Urbanecm_WMF created this task.Sep 23 2021, 6:23 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 23 2021, 6:23 PM
Urbanecm_WMF triaged this task as High priority.Sep 23 2021, 6:23 PM
Urbanecm_WMF added a parent task: T277862: selenium-daily-beta-WikibaseLexeme broken since mid feb 2021.
Urbanecm_WMF moved this task from Incoming to Ready for Development on the Growth-Team (Sprint 0 (Growth Team)) board.
gerritbot added a comment.Sep 23 2021, 6:35 PM

Change 723296 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] updateMenteeData.php: Decrease number of queries made

https://gerrit.wikimedia.org/r/723296

gerritbot added a project: Patch-For-Review.Sep 23 2021, 6:35 PM
Urbanecm_WMF moved this task from Ready for Development to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Sep 23 2021, 6:40 PM
Urbanecm_WMF moved this task from Backlog to In review on the User-Urbanecm_WMF (Engineering) board.
gerritbot added a comment.Sep 24 2021, 1:39 AM

Change 723296 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] updateMenteeData.php: Decrease number of write queries made

https://gerrit.wikimedia.org/r/723296

ReleaseTaggerBot added a project: MW-1.38-notes (1.38.0-wmf.2; 2021-09-28).Sep 24 2021, 2:00 AM
Maintenance_bot removed a project: Patch-For-Review.Sep 24 2021, 2:10 AM
gerritbot added a comment.Sep 27 2021, 1:46 PM

Change 723782 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@wmf/1.38.0-wmf.1] updateMenteeData.php: Decrease number of write queries made

https://gerrit.wikimedia.org/r/723782

gerritbot added a project: Patch-For-Review.Sep 27 2021, 1:46 PM
gerritbot added a comment.Sep 27 2021, 2:08 PM

Change 723782 merged by Urbanecm:

[mediawiki/extensions/GrowthExperiments@wmf/1.38.0-wmf.1] updateMenteeData.php: Decrease number of write queries made

https://gerrit.wikimedia.org/r/723782

Maintenance_bot removed a project: Patch-For-Review.Sep 27 2021, 2:10 PM
Stashbot added a comment.Sep 27 2021, 2:36 PM

Mentioned in SAL (#wikimedia-operations) [2021-09-27T14:36:55Z] <urbanecm@deploy1002> Synchronized php-1.38.0-wmf.1/extensions/GrowthExperiments/: 08f1e73: 3b154db: GrowthExperiments backports (T290609, T291658) (duration: 00m 58s)

Stashbot mentioned this in T290609: Make mentee overview module's updateMenteeData.php scale better.Sep 27 2021, 2:36 PM
ReleaseTaggerBot edited projects, added MW-1.38-notes (1.38.0-wmf.1; 2021-09-21); removed MW-1.38-notes (1.38.0-wmf.2; 2021-09-28).Sep 27 2021, 3:00 PM
Urbanecm_WMF closed this task as Resolved.Sep 27 2021, 4:35 PM
Urbanecm_WMF mentioned this in T291836: updateMenteeData.php runs weird queries.

Verified in beta's binlogs, looks okay now.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL