Page MenuHomePhabricator
Create Task
Maniphest T194164

Start reading from change_tag_def in production
Closed, ResolvedPublic
Actions

Description

On some date (to be defined), set $wgChangeTagsSchemaMigrationStage to MIGRATION_NEW in order to start reading from the new change_tag_def table.

Details

SubjectRepoBranchLines +/-
Enable reading from new backend of change tag everywhereoperations/mediawiki-configmaster+1 -18
Avoid fatals when the filter tags is emptymediawiki/corewmf/1.32.0-wmf.24+4 -1
Avoid fatals when the filter tags is emptymediawiki/coremaster+4 -1
Enable reading from new backend of change_tag in s7operations/mediawiki-configmaster+12 -1
Enable reading from ct_tag_id in s7operations/mediawiki-configmaster+1 -0
Set some small wikis to read new for change tag backendoperations/mediawiki-configmaster+5 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

Lucas_Werkmeister_WMDE created this task.May 8 2018, 2:24 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 8 2018, 2:24 PM
Lucas_Werkmeister_WMDE mentioned this in T185355: Normalize change tag schema.May 8 2018, 2:27 PM
Lucas_Werkmeister_WMDE added a project: Wikidata-Campsite.Jun 5 2018, 1:35 PM
• Vvjjkkii renamed this task from Start reading from change_tag_def in production to vddaaaaaaa.Jul 1 2018, 1:11 AM
• Vvjjkkii triaged this task as High priority.
• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).
• Vvjjkkii updated the task description. (Show Details)
• Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot renamed this task from vddaaaaaaa to Start reading from change_tag_def in production.Jul 1 2018, 3:31 PM
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.
CommunityTechBot added a subscriber: Aklapper.
Krinkle removed a project: MediaWiki-libs-Rdbms.Jul 28 2018, 10:39 PM
Lydia_Pintscher closed subtask T196671: Start reading from change_tag_def in beta cluster as Resolved.Sep 16 2018, 10:37 AM
gerritbot subscribed.Oct 11 2018, 10:00 AM

Change 466271 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Set some small wikis to read new for change tag backend

https://gerrit.wikimedia.org/r/466271

Restricted Application added a project: Wikidata. · View Herald TranscriptOct 11 2018, 10:00 AM
gerritbot added a project: Patch-For-Review.Oct 11 2018, 10:00 AM
Ladsgroup claimed this task.Oct 11 2018, 10:00 AM
Ladsgroup edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed Wikidata-Ministry-Of-Magic-Tech-Debt, Wikidata-Ministry-Of-Magic.
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptOct 11 2018, 10:00 AM
Ladsgroup moved this task from To Do (prioritised from top to bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Oct 11 2018, 10:01 AM
gerritbot added a comment.Oct 11 2018, 11:13 AM

Change 466271 merged by jenkins-bot:
[operations/mediawiki-config@master] Set some small wikis to read new for change tag backend

https://gerrit.wikimedia.org/r/466271

Stashbot subscribed.Oct 11 2018, 11:30 AM

Mentioned in SAL (#wikimedia-operations) [2018-10-11T11:30:41Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:466271|Set some small wikis to read new for change tag backend (T194164)]] (duration: 00m 50s)

gerritbot added a comment.Oct 15 2018, 10:46 AM

Change 467307 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Enable reading from ct_tag_id in s7

https://gerrit.wikimedia.org/r/467307

gerritbot added a comment.Oct 15 2018, 11:07 AM

Change 467307 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable reading from ct_tag_id in s7

https://gerrit.wikimedia.org/r/467307

Stashbot added a comment.Oct 15 2018, 11:09 AM

Mentioned in SAL (#wikimedia-operations) [2018-10-15T11:09:32Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:467307|Enable reading from ct_tag_id in s7 (T194164)]] (duration: 00m 49s)

gerritbot added a comment.Oct 15 2018, 11:35 AM

Change 467315 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Enable reading from new backend of change_tag in s7

https://gerrit.wikimedia.org/r/467315

Addshore triaged this task as Medium priority.Oct 15 2018, 12:28 PM
gerritbot added a comment.Oct 15 2018, 6:52 PM

Change 467315 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable reading from new backend of change_tag in s7

https://gerrit.wikimedia.org/r/467315

Stashbot added a comment.Oct 15 2018, 6:59 PM

Mentioned in SAL (#wikimedia-operations) [2018-10-15T18:59:48Z] <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:467315|Enable reading from new backend of change_tag in s7 (T194164)]] (duration: 00m 49s)

Legoktm subscribed.EditedOct 16 2018, 6:27 AM

Going to revert because of:

2018-10-16 06:26:29 [W8WElQpAMFcAAKy-ox4AAAIY] mw1252 hewiki 1.32.0-wmf.24 exception ERROR: [W8WElQpAMFcAAKy-ox4AAAIY] /w/index.php?namespace=lWCq&invert=1%27%3BSELECT%20%28CASE%20WHEN%20%285897%3D5897%29%20THEN%205897%20ELSE%201%2
F%28SELECT%200%29%20END%29--&associated=1&tagfilter=Uvhq&target=%D7%A2%D7%9E%D7%95%D7%93%20%D7%A8%D7%90%D7%A9%D7%99&showlinkedto=1&title=%D7%9E%D7%99%D7%95%D7%97%D7%93%3A%D7%A9%D7%99%D7%A0%D7%95%D7%99%D7%99%D7%9D%20%D7%91%D7%93%D7%
A4%D7%99%D7%9D%20%D7%94%D7%9E%D7%A7%D7%95%D7%A9%D7%A8%D7%99%D7%9D   InvalidArgumentException from line 2139 of /srv/mediawiki/php-1.32.0-wmf.24/includes/libs/rdbms/database/Database.php: Wikimedia\Rdbms\Database::makeList: empty in
put for field ct_tag_id {"exception_id":"W8WElQpAMFcAAKy-ox4AAAIY","exception_url":"/w/index.php?namespace=lWCq&invert=1%27%3BSELECT%20%28CASE%20WHEN%20%285897%3D5897%29%20THEN%205897%20E
LSE%201%2F%28SELECT%200%29%20END%29--&associated=1&tagfilter=Uvhq&target=%D7%A2%D7%9E%D7%95%D7%93%20%D7%A8%D7%90%D7%A9%D7%99&showlinkedto=1&title=%D7%9E%D7%99%D7%95%D7%97%D7%93%3A%D7%A9%D7%99%D7%A0%D7%95%D7%99%D7%99%D7%9D%20%D7%91%D7%93%D7%A4%D7%99%D7%9D%20%D7%94%D7%9E%D7%A7%D7%95%D7%A9%D7%A8%D7%99%D7%9D","caught_by":"mwe_handler"} 
[Exception InvalidArgumentException] (/srv/mediawiki/php-1.32.0-wmf.24/includes/libs/rdbms/database/Database.php:2139) Wikimedia\Rdbms\Database::makeList: empty input for field ct_tag_id
  #0 /srv/mediawiki/php-1.32.0-wmf.24/includes/libs/rdbms/database/Database.php(1732): Wikimedia\Rdbms\Database->makeList(array, integer)
  #1 /srv/mediawiki/php-1.32.0-wmf.24/includes/specials/SpecialRecentchangeslinked.php(211): Wikimedia\Rdbms\Database->selectSQLText(array, array, array, string, array, array)
  #2 /srv/mediawiki/php-1.32.0-wmf.24/includes/specialpage/ChangesListSpecialPage.php(934): SpecialRecentChangesLinked->doMainQuery(array, array, array, array, array, FormOptions)
  #3 /srv/mediawiki/php-1.32.0-wmf.24/includes/specialpage/ChangesListSpecialPage.php(629): ChangesListSpecialPage->getRows()
  #4 /srv/mediawiki/php-1.32.0-wmf.24/includes/specials/SpecialRecentchanges.php(167): ChangesListSpecialPage->execute(NULL)
  #5 /srv/mediawiki/php-1.32.0-wmf.24/includes/specialpage/SpecialPage.php(569): SpecialRecentChanges->execute(NULL)
  #6 /srv/mediawiki/php-1.32.0-wmf.24/includes/specialpage/SpecialPageFactory.php(568): SpecialPage->run(NULL)
  #7 /srv/mediawiki/php-1.32.0-wmf.24/includes/MediaWiki.php(288): MediaWiki\Special\SpecialPageFactory->executePath(Title, RequestContext)
  #8 /srv/mediawiki/php-1.32.0-wmf.24/includes/MediaWiki.php(868): MediaWiki->performRequest()
  #9 /srv/mediawiki/php-1.32.0-wmf.24/includes/MediaWiki.php(525): MediaWiki->main()
  #10 /srv/mediawiki/php-1.32.0-wmf.24/index.php(42): MediaWiki->run()
  #11 /srv/mediawiki/w/index.php(3): include(string)
  #12 {main}

Happening about 200/minute per Joe.

Stashbot added a comment.Oct 16 2018, 6:32 AM

Mentioned in SAL (#wikimedia-operations) [2018-10-16T06:32:09Z] <legoktm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Revert "Enable reading from new backend of change_tag in s7" (T194164) (duration: 00m 50s)

Marostegui subscribed.Oct 16 2018, 6:35 AM
Nikerabbit subscribed.Oct 16 2018, 6:53 AM
Tgr subscribed.Oct 16 2018, 7:11 AM

Seems to have started an hour ago, though, not when the patch was deployed: https://logstash.wikimedia.org/goto/ed4d3af593dd6d272ee6b5ef0bed5d67

Tgr added a comment.Oct 16 2018, 7:13 AM

And all the URLs are crazy like that. Is that some kind of SQL injection scanning?

Ladsgroup added a comment.Oct 16 2018, 9:34 AM
In T194164#4669366, @Tgr wrote:

And all the URLs are crazy like that. Is that some kind of SQL injection scanning?

Yes, they are trying to scan for SQL injections and that's why it started fatal 18 hours after deploying the change (exactly when I was asleep). We should ban them IMO but still it should not trigger fatals. I'm on it.

gerritbot added a comment.Oct 16 2018, 11:04 AM

Change 467666 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/core@master] Avoid fatals when the filter tags is empty

https://gerrit.wikimedia.org/r/467666

gerritbot added a comment.Oct 16 2018, 11:30 AM

Change 467670 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/core@wmf/1.32.0-wmf.24] Avoid fatals when the filter tags is empty

https://gerrit.wikimedia.org/r/467670

gerritbot added a comment.Oct 16 2018, 11:36 AM

Change 467666 merged by jenkins-bot:
[mediawiki/core@master] Avoid fatals when the filter tags is empty

https://gerrit.wikimedia.org/r/467666

gerritbot added a comment.Oct 16 2018, 11:57 AM

Change 467670 merged by Ladsgroup:
[mediawiki/core@wmf/1.32.0-wmf.24] Avoid fatals when the filter tags is empty

https://gerrit.wikimedia.org/r/467670

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-10-16 (1.32.0-wmf.26)).Oct 16 2018, 12:00 PM
Stashbot added a comment.Oct 16 2018, 12:03 PM

Mentioned in SAL (#wikimedia-operations) [2018-10-16T12:03:08Z] <ladsgroup@deploy1001> Synchronized php-1.32.0-wmf.24/includes/changetags/ChangeTags.php: SWAT: [[gerrit:467670|Avoid fatals when the filter tags is empty (T194164)]] (duration: 00m 50s)

Stashbot added a comment.Oct 16 2018, 12:08 PM

Mentioned in SAL (#wikimedia-operations) [2018-10-16T12:08:33Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:467669|Enable reading from new backend of change_tag in s7 (T194164)]] (duration: 00m 50s)

Addshore moved this task from incoming to in progress on the Wikidata board.Oct 16 2018, 1:49 PM
gerritbot added a comment.Oct 17 2018, 10:51 AM

Change 467946 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] Enable reading from new backend of change tag everywhere

https://gerrit.wikimedia.org/r/467946

gerritbot added a comment.Oct 17 2018, 11:33 AM

Change 467946 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable reading from new backend of change tag everywhere

https://gerrit.wikimedia.org/r/467946

Stashbot added a comment.Oct 17 2018, 11:40 AM

Mentioned in SAL (#wikimedia-operations) [2018-10-17T11:40:01Z] <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:467946|Enable reading from new backend of change tag everywhere (T194164)]] (duration: 00m 57s)

Ladsgroup moved this task from Doing to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Oct 17 2018, 11:44 AM
Ladsgroup mentioned this in T207313: Some administrative and log actions on Wikidata take longer than 60 seconds and time out.Oct 19 2018, 11:07 AM
Addshore removed a project: Wikidata-Campsite.Nov 1 2018, 11:21 AM
Addshore closed this task as Resolved.Nov 6 2018, 12:13 PM
Addshore moved this task from Test (Verification) to Done on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
Addshore subscribed.

The "start" as described in this ticket is done, I'll file a followup for wikidata.org which was rolled back so it is still not done there yet.

Addshore mentioned this in T208846: Start reading from change_tag_def on wikidatawiki.Nov 6 2018, 12:15 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits