Page MenuHomePhabricator
Create Task
Maniphest T385645

Drop event_variant column from echo_event
Closed, ResolvedPublic
Actions

Description

  1. ALTERs to run: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Echo/+/1111294/1/sql/mysql/patch-echo_event-event_variant.sql
  2. Where to run those changes: x1, all dbs
  3. When to run those changes: When DBA has time
  4. If the schema change is backwards compatible: No
  5. If the schema change has been tested already on some of the test/beta wikis: Already in beta cluster.
  6. If the data should be made available on the labs replicas and/or dumps: Nope, nada, nothing.

Other "special" wikis not under x1:

  • metawiki s7
  • officewiki s3
  • mediawikiwiki s3

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
x1: Change format to STATEMENToperations/puppetproduction+8 -9
db1179: Disable notificationsoperations/puppetproduction+1 -0
db1237: Disable notificationsoperations/puppetproduction+1 -1
Customize query in gerrit

Related Objects

Event Timeline

Ladsgroup created this task.Feb 4 2025, 10:19 PM
Restricted Application added a project: Data-Engineering. · View Herald TranscriptFeb 4 2025, 10:19 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Marostegui claimed this task.Feb 5 2025, 6:07 AM
Marostegui triaged this task as Medium priority.
Marostegui moved this task from Triage to Ready on the DBA board.
CodeReviewBot added a project: Patch-For-Review.Feb 5 2025, 9:52 AM

marostegui opened https://gitlab.wikimedia.org/repos/sre/schema-changes/-/merge_requests/34

odrop_event_variant_T385645.py: New schema change

Stashbot added a comment.Feb 5 2025, 10:26 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-05T10:26:51Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1179 (T385645)', diff saved to https://phabricator.wikimedia.org/P73210 and previous config saved to /var/cache/conftool/dbconfig/20250205-102650-marostegui.json

Marostegui added a comment.Feb 5 2025, 10:26 AM

Running in eqiad

Stashbot added a comment.Feb 5 2025, 10:27 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-05T10:27:58Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T385645)', diff saved to https://phabricator.wikimedia.org/P73211 and previous config saved to /var/cache/conftool/dbconfig/20250205-102758-marostegui.json

Marostegui added a comment.Feb 5 2025, 10:42 AM

I forgot x1 is RBR, so we need to make this SBR for this schema change to be applied on replicas first.

Stashbot added a comment.Feb 5 2025, 10:43 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-05T10:43:46Z] <marostegui> Set x1 to SBR for a bit T385645

Marostegui mentioned this in rSCHCHfaa258aeadca: odrop_event_variant_T385645.py: New schema change.Feb 5 2025, 10:59 AM
gerritbot added a comment.Feb 5 2025, 11:46 AM

Change #1117517 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] x1: Change format to STATEMENT

https://gerrit.wikimedia.org/r/1117517

gerritbot added a comment.Feb 5 2025, 11:47 AM

Change #1117517 merged by Marostegui:

[operations/puppet@production] x1: Change format to STATEMENT

https://gerrit.wikimedia.org/r/1117517

gerritbot added a comment.Feb 5 2025, 12:48 PM

Change #1117532 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] db1179: Disable notifications

https://gerrit.wikimedia.org/r/1117532

gerritbot added a comment.Feb 5 2025, 12:48 PM

Change #1117532 merged by Marostegui:

[operations/puppet@production] db1179: Disable notifications

https://gerrit.wikimedia.org/r/1117532

Marostegui added a comment.Feb 5 2025, 12:49 PM

@Ladsgroup looks like there is still things writing to this column:

Last_SQL_Error: Error 'Unknown column 'event_variant' in 'field list'' on query. Default database: 'bnwiki'. Query: 'INSERT /* MediaWiki\Extension\Notifications\Mapper\EventMapper::insert  */ INTO `echo_event` (event_type,event_variant,event_deleted,event_extra,event_agent_id,event_page_id) VALUES ('page-linked',NULL,0,'a:3:{s:11:\"target-page\";i:1511611;s:17:\"link-from-page-id\";i:1511611;s:5:\"revid\";i:7951227;}',443392,80084)'
gerritbot added a comment.Feb 5 2025, 12:51 PM

Change #1117533 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] db1237: Disable notifications

https://gerrit.wikimedia.org/r/1117533

gerritbot added a comment.Feb 5 2025, 12:52 PM

Change #1117533 merged by Marostegui:

[operations/puppet@production] db1237: Disable notifications

https://gerrit.wikimedia.org/r/1117533

Marostegui changed the task status from Open to Stalled.Feb 5 2025, 5:37 PM
Ladsgroup mentioned this in T380449: Optimize two echo tables in x1.Feb 10 2025, 11:23 AM
Ottomata moved this task from Incoming (new tickets) to Tag with Radar on the Data-Engineering board.Feb 11 2025, 6:57 PM
Ottomata added a project: Data-Engineering-Radar.
Stashbot added a comment.Feb 25 2025, 10:03 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-25T10:03:34Z] <marostegui> Move x1 back to RBR T385645

Marostegui added a comment.Feb 25 2025, 10:04 AM

x1 moved back to RBR until T385645#10525207 is fixed!

Ladsgroup changed the task status from Stalled to Open.Feb 25 2025, 11:07 AM

The code should have reached production now.

Stashbot added a comment.Feb 25 2025, 11:39 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-25T11:39:28Z] <marostegui> Deploy schema change on x1 db1179 eqiad dbmaint T385645

Marostegui added a comment.Feb 25 2025, 12:55 PM

db1179 has been altered. I am leaving it for a few more hours to make sure it is good.

Marostegui moved this task from Ready to In progress on the DBA board.Feb 26 2025, 6:54 AM
Stashbot added a comment.Feb 26 2025, 6:56 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-26T06:56:35Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1179 (T385645)', diff saved to https://phabricator.wikimedia.org/P73609 and previous config saved to /var/cache/conftool/dbconfig/20250226-065634-marostegui.json

Stashbot added a comment.Feb 26 2025, 6:57 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-26T06:57:43Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T385645)', diff saved to https://phabricator.wikimedia.org/P73610 and previous config saved to /var/cache/conftool/dbconfig/20250226-065742-marostegui.json

Stashbot added a comment.Feb 26 2025, 7:43 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-26T07:43:10Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T385645)', diff saved to https://phabricator.wikimedia.org/P73621 and previous config saved to /var/cache/conftool/dbconfig/20250226-074309-marostegui.json

Stashbot added a comment.Feb 26 2025, 7:43 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-26T07:43:48Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1224 (T385645)', diff saved to https://phabricator.wikimedia.org/P73622 and previous config saved to /var/cache/conftool/dbconfig/20250226-074347-marostegui.json

Stashbot added a comment.Feb 26 2025, 7:44 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-26T07:44:56Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224 (T385645)', diff saved to https://phabricator.wikimedia.org/P73623 and previous config saved to /var/cache/conftool/dbconfig/20250226-074455-marostegui.json

Stashbot added a comment.Feb 26 2025, 7:53 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-26T07:53:48Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1224 T385645', diff saved to https://phabricator.wikimedia.org/P73625 and previous config saved to /var/cache/conftool/dbconfig/20250226-075347-root.json

CodeReviewBot added a comment.Feb 26 2025, 10:25 AM

marostegui merged https://gitlab.wikimedia.org/repos/sre/schema-changes/-/merge_requests/34

Drop_event_variant_T385645.py: New schema change

Maintenance_bot removed a project: Patch-For-Review.Feb 26 2025, 10:30 AM
Ladsgroup added a comment.Feb 26 2025, 10:52 AM

Three wikis (metawiki, mediawikiwiki, officewiki, check wmgEchoCluster) have their Echo tables in core db instead of x1. Which means they probably haven't got the many other previous schema changes either.

Marostegui added a comment.Feb 26 2025, 10:54 AM
In T385645#10582080, @Ladsgroup wrote:

Three wikis (metawiki, mediawikiwiki, officewiki, check wmgEchoCluster) have their Echo tables in core db instead of x1. Which means they probably haven't got the many other previous schema changes either.

This is a very good reminder - I will add it on the task so it is not forgotten

Marostegui updated the task description. (Show Details)Feb 26 2025, 10:55 AM
jcrespo subscribed.Feb 26 2025, 10:55 AM

This is T119154. We could reconsider reopening if it makes things better in the long term, if the effort is worth the payback. Up to the people here.

Stashbot added a comment.Feb 26 2025, 10:59 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-26T10:59:53Z] <marostegui> Drop schema change on s3 codfw master with replication dbmaint T385645

Marostegui updated the task description. (Show Details)Feb 26 2025, 11:00 AM
Stashbot added a comment.Feb 26 2025, 11:03 AM

Mentioned in SAL (#wikimedia-operations) [2025-02-26T11:03:37Z] <marostegui> Drop schema change on s7 codfw master with replication dbmaint T385645

Marostegui updated the task description. (Show Details)Feb 26 2025, 11:04 AM
Ladsgroup added a comment.Feb 26 2025, 11:06 AM
In T385645#10582106, @jcrespo wrote:

This is T119154. We could reconsider reopening if it makes things better in the long term, if the effort is worth the payback. Up to the people here.

I actually fixed wikitech yesterday but it was a disruptive one (but that's fine, wikitech is not a top tier content wiki) but for others it's gonna be painful.

Marostegui mentioned this in rSCHCH6cb21db74d60: Merge branch 'T385645' into 'main'.Feb 26 2025, 12:17 PM
Marostegui added a comment.Feb 27 2025, 6:12 AM

This is done, so I am going to revert back to RBR

Marostegui closed this task as Resolved.Feb 27 2025, 6:18 AM

All done

Maintenance_bot moved this task from In progress to Done on the DBA board.Feb 27 2025, 6:29 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits