Page MenuHomePhabricator

Remove revision_comment_temp and revision_actor_temp
Closed, ResolvedPublic

Description

Due to expected difficulty in altering the revision table (see T161671: Compacting the revision table), the actor and comment migrations created tables named revision_comment_temp and revision_actor_temp to relate the rev_id and the comment_id/actor_id, with the plan being to eventually alter revision to contain rev_comment_id and rev_actor columns and drop those relation tables. This task tracks that "eventually".

  • 0. Wait for the main actor and comment migration to be done (except for DBA work).
    • 0.1. Comment migration, through steps 7.1 and 7.2 (T166733)
    • 0.2. Actor migration, through steps 7.1 and 7.2 (T188327)
  • 1. Alter revision to drop rev_comment, rev_user, and rev_user_text and add rev_comment_id and rev_actor. (T161671)
    • 1.1. Write and merge the MediaWiki patch.
    • 1.2. Have DBAs perform the change.
  • 2. Introduce a migration from using revision_comment_temp and revision_actor_temp (old) to rev_comment_id and rev_actor (new).
    • 2.1. Write and merge the MediaWiki patch, assuming it wasn't included in 1.1.
    • 2.2. Check for extensions assuming the temp tables exist.
  • 3. Set migration stage to write-both/read-old.
  • 4. Run the maintenance script to backfill rev_comment_id and rev_actor.
  • 5. Set migration stage to write-both/read-new.
    • 5.1. Announce changes to wikitech-l and cloud. not necessary, since the tables cannot be queried directly.
    • 5.2. Update WMCS views to read the new schema.
  • 6. Set migration stage to write-new/read-new.
  • 7. Remove old tables and code.

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Next steps:

  1. Write to rev_actor and rev_comment_id
  2. Backfill rev_actor and rev_comment_id
  3. have a read-new mode, at least for testing
  • We won’t get through with the migration before the 1.36 branch (March?)
    • We should avoid supporting multiple modes in a release
  • But we can start writing to the new schema, and we can also already backfill.
  • The Updater to 1.36 should also backfill.

@Marostegui is it correct that step 1.2 of this task is already completed by DBAs? Can we mark step 1 done?

Correct, this is how the table looks like now:

root@cumin1001:/home/marostegui# mysql.py -hdb1169 enwiki -e "show create table revision\G"
*************************** 1. row ***************************
       Table: revision
Create Table: CREATE TABLE `revision` (
  `rev_id` int(8) unsigned NOT NULL AUTO_INCREMENT,
  `rev_page` int(8) unsigned NOT NULL DEFAULT 0,
  `rev_comment_id` bigint(20) unsigned NOT NULL DEFAULT 0,
  `rev_actor` bigint(20) unsigned NOT NULL DEFAULT 0,
  `rev_timestamp` varbinary(14) NOT NULL DEFAULT '',
  `rev_minor_edit` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `rev_deleted` tinyint(1) unsigned NOT NULL DEFAULT 0,
  `rev_len` int(8) unsigned DEFAULT NULL,
  `rev_parent_id` int(8) unsigned DEFAULT NULL,
  `rev_sha1` varbinary(32) NOT NULL DEFAULT '',
  PRIMARY KEY (`rev_id`),
  KEY `rev_timestamp` (`rev_timestamp`),
  KEY `page_timestamp` (`rev_page`,`rev_timestamp`),
  KEY `rev_page_id` (`rev_page`,`rev_id`),
  KEY `rev_actor_timestamp` (`rev_actor`,`rev_timestamp`,`rev_id`),
  KEY `rev_page_actor_timestamp` (`rev_page`,`rev_actor`,`rev_timestamp`)
) ENGINE=InnoDB AUTO_INCREMENT=1009766990 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED

Change 793845 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] Start clean up of revision_actor_table

https://gerrit.wikimedia.org/r/793845

Change 793845 merged by jenkins-bot:

[mediawiki/core@master] Start clean up of revision_actor_temp table

https://gerrit.wikimedia.org/r/793845

Change 830204 had a related patch set uploaded (by Umherirrender; author: Umherirrender):

[mediawiki/core@master] Remove references to revision_actor_temp, aliased as temp_rev_user

https://gerrit.wikimedia.org/r/830204

Change 830204 merged by jenkins-bot:

[mediawiki/core@master] Remove references to revision_actor_temp, aliased as temp_rev_user

https://gerrit.wikimedia.org/r/830204

Change 873880 had a related patch set uploaded (by Zabe; author: Zabe):

[operations/mediawiki-config@master] Stop setting $wgActorTableSchemaMigrationStage

https://gerrit.wikimedia.org/r/873880

Change 873882 had a related patch set uploaded (by Zabe; author: Zabe):

[operations/software@master] tables_to_check: drop revision_actor_temp

https://gerrit.wikimedia.org/r/873882

Change 873882 merged by jenkins-bot:

[operations/software@master] tables_to_check: drop revision_actor_temp

https://gerrit.wikimedia.org/r/873882

Change 874466 had a related patch set uploaded (by Func; author: Func):

[mediawiki/core@REL1_39] MigrateActors: Write to revsion table (Follow-up 24115a8)

https://gerrit.wikimedia.org/r/874466

Change 874466 merged by jenkins-bot:

[mediawiki/core@REL1_39] MigrateActors: Write to revsion table (Follow-up 24115a8)

https://gerrit.wikimedia.org/r/874466

Change 873880 merged by jenkins-bot:

[operations/mediawiki-config@master] Stop setting $wgActorTableSchemaMigrationStage

https://gerrit.wikimedia.org/r/873880

Mentioned in SAL (#wikimedia-operations) [2023-01-03T21:06:40Z] <taavi@deploy1002> Started scap: Backport for [[gerrit:873880|Stop setting $wgActorTableSchemaMigrationStage (T215466)]], [[gerrit:873887|Pin $wgCommentTempTableSchemaMigrationStage to default value (T299954)]], [[gerrit:874418|Pin cu_changes comment migration to old schema (T233004)]]

Mentioned in SAL (#wikimedia-operations) [2023-01-03T21:08:30Z] <taavi@deploy1002> taavi and zabe: Backport for [[gerrit:873880|Stop setting $wgActorTableSchemaMigrationStage (T215466)]], [[gerrit:873887|Pin $wgCommentTempTableSchemaMigrationStage to default value (T299954)]], [[gerrit:874418|Pin cu_changes comment migration to old schema (T233004)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-01-03T21:15:29Z] <taavi@deploy1002> Finished scap: Backport for [[gerrit:873880|Stop setting $wgActorTableSchemaMigrationStage (T215466)]], [[gerrit:873887|Pin $wgCommentTempTableSchemaMigrationStage to default value (T299954)]], [[gerrit:874418|Pin cu_changes comment migration to old schema (T233004)]] (duration: 08m 49s)

Change 874878 had a related patch set uploaded (by Func; author: Func):

[mediawiki/core@REL1_39] installer: Split drop action out of the SQL patch for actor migration

https://gerrit.wikimedia.org/r/874878

Change 874878 merged by jenkins-bot:

[mediawiki/core@REL1_39] installer: Split drop action out of the SQL patch for actor migration

https://gerrit.wikimedia.org/r/874878

Zabe updated the task description. (Show Details)

Change 924122 had a related patch set uploaded (by Zabe; author: Zabe):

[operations/software@master] tables_to_check: drop revision_comment_temp

https://gerrit.wikimedia.org/r/924122

Change 924122 merged by jenkins-bot:

[operations/software@master] tables_to_check: drop revision_comment_temp

https://gerrit.wikimedia.org/r/924122

Change 924468 had a related patch set uploaded (by Zabe; author: Zabe):

[mediawiki/core@master] Drop default from rev_actor and rev_comment_id

https://gerrit.wikimedia.org/r/924468

Change 925547 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/puppet@production] wmcs: Update the revision_comment_temp index to revision table

https://gerrit.wikimedia.org/r/925547

Change 925547 merged by Ladsgroup:

[operations/puppet@production] wmcs: Update the revision_comment_temp index to revision table

https://gerrit.wikimedia.org/r/925547

Running maintain-views and maintain-indexes on clouddb hosts now.

Change 929717 had a related patch set uploaded (by Zabe; author: Zabe):

[mediawiki/core@master] CommentStore: Drop temp table code

https://gerrit.wikimedia.org/r/929717

Change 929720 had a related patch set uploaded (by Zabe; author: Zabe):

[mediawiki/core@master] Stop using CommentStoreBase::insertWithTempTable for rev_comment

https://gerrit.wikimedia.org/r/929720

Change 929720 abandoned by Zabe:

[mediawiki/core@master] Stop using CommentStoreBase::insertWithTempTable for rev_comment

Reason:

merging into https://gerrit.wikimedia.org/r/c/mediawiki/core/+/929717

https://gerrit.wikimedia.org/r/929720

Change 929717 merged by jenkins-bot:

[mediawiki/core@master] CommentStore: Drop temp table code

https://gerrit.wikimedia.org/r/929717

Change 991670 had a related patch set uploaded (by Zabe; author: Zabe):

[mediawiki/core@master] ActorMigration: Drop temp table code

https://gerrit.wikimedia.org/r/991670

Change 991670 merged by jenkins-bot:

[mediawiki/core@master] ActorMigration: Drop temp table code

https://gerrit.wikimedia.org/r/991670

Change #924468 merged by jenkins-bot:

[mediawiki/core@master] Cleanup revision table schema

https://gerrit.wikimedia.org/r/924468