Page MenuHomePhabricator

Populate rev_actor and rev_comment_id
Open, Needs TriagePublic1 Estimated Story Points

Description

This includes steps 2-4 of T215466:

  • 2. Introduce a migration from using revision_comment_temp and revision_actor_temp (old) to rev_comment_id and rev_actor (new).
    • 2.1. Write and merge the MediaWiki patch, assuming it wasn't included in 1.1.
    • 2.2. Check for extensions assuming the temp tables exist.
  • 3. Set migration stage to write-both/read-old.
  • 4. Run the maintenance script to backfill rev_comment_id and rev_actor.

From T215466#6803173:

Next steps:

  1. Write to rev_actor and rev_comment_id
  2. Backfill rev_actor and rev_comment_id
  3. have a read-new mode, at least for testing
  • We won’t get through with the migration before the 1.36 branch (March?)
    • We should avoid supporting multiple modes in a release
  • But we can start writing to the new schema, and we can also already backfill.
  • The Updater to 1.36 should also backfill.

Event Timeline

There's a lot of cleanup of deprecated code which needs to be done first.

There are a very large number of ActorMigration callers using soft-deprecated key fields with names that no longer exist in the schema. That's what I'm working on first. The references will be removed and these usages will become hard-deprecated.

Change 674997 had a related patch set uploaded (by Tim Starling; author: Tim Starling):
[mediawiki/core@master] [WIP] Deprecate legacy ActorMigration fields and stop using them

https://gerrit.wikimedia.org/r/674997

I'm splitting out the cleanup task to T278917.

We discussed this in the Expedition meeting and believe it can be pushed to 1.37. @tstarling, let us know if you think there's anything that does need to get into 1.36. It would have to be merged this week.

Change 684142 had a related patch set uploaded (by Tim Starling; author: Tim Starling):

[mediawiki/core@master] [WIP] Schema migration for revison_actor_temp table removal

https://gerrit.wikimedia.org/r/684142

Change 701218 had a related patch set uploaded (by Tim Starling; author: Tim Starling):

[mediawiki/extensions/FlaggedRevs@master] Fix the name of the new contributions index

https://gerrit.wikimedia.org/r/701218

Change 684142 merged by jenkins-bot:

[mediawiki/core@master] Schema migration for revison_actor_temp table removal

https://gerrit.wikimedia.org/r/684142

Hey there. This task is proposed as a blocker to MediaWiki 1.37, which will be cut in less than three weeks' time. Please consider whether this will make that deadline, and if not, move it to block the MediaWiki 1.38 release (MW-1.38-release) or remove as a blocker entirely.

Hi, what's the status of this? I would like to finish the migration and if there is anything I can help, let me know.

I updated the checklist. The next step is to do the migration in production. Wikidata reads revision rows from foreign databases, which implies that the configuration for all wikis should be kept in sync. So:

  • Set $wgActorTableSchemaMigrationStage = SCHEMA_COMPAT_WRITE_TEMP_AND_NEW | SCHEMA_COMPAT_READ_TEMP
  • Run maintenance/migrateRevisionActorTemp.php on all wikis
  • Set $wgActorTableSchemaMigrationStage = SCHEMA_COMPAT_WRITE_TEMP_AND_NEW | SCHEMA_COMPAT_READ_NEW

I think that will resolve this task. The task description says that the migration should also be added to the updater, but I think that can wait until after production has been migrated, i.e. step 7 of T215466, on the principle that we should fully test our code in production before forcing non-WMF users to use it.

Thanks. I will do the switching the config on Monday and start the backfill script in test wikis (then small wikis and so on). I hope that's fine with you.

Change 739633 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):

[operations/mediawiki-config@master] Revert \"Stop setting wgActorTableSchemaMigrationStage, no longer read in core\"

https://gerrit.wikimedia.org/r/739633

Change 739633 merged by jenkins-bot:

[operations/mediawiki-config@master] Revert \"Stop setting wgActorTableSchemaMigrationStage, no longer read in core\"

https://gerrit.wikimedia.org/r/739633

Mentioned in SAL (#wikimedia-operations) [2021-11-18T01:35:29Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: NOOP - Config: [[gerrit:739633|Revert "Stop setting wgActorTableSchemaMigrationStage, no longer read in core" (T275246)]] (duration: 01m 04s)

Mentioned in SAL (#wikimedia-releng) [2021-11-18T01:42:21Z] <Amir1> ladsgroup@deployment-deploy01:~$ foreachwikiindblist all-labs maintenance/migraeRevisionActorTemp.php (T275246)

Change 739695 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] maintenance: Add waitForReplication and sleep in migrateRevisionActorTemp

https://gerrit.wikimedia.org/r/739695

Change 739695 merged by jenkins-bot:

[mediawiki/core@master] maintenance: Add waitForReplication and sleep in migrateRevisionActorTemp

https://gerrit.wikimedia.org/r/739695

Change 739636 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@wmf/1.38.0-wmf.9] maintenance: Add waitForReplication and sleep in migrateRevisionActorTemp

https://gerrit.wikimedia.org/r/739636

Change 739636 merged by jenkins-bot:

[mediawiki/core@wmf/1.38.0-wmf.9] maintenance: Add waitForReplication and sleep in migrateRevisionActorTemp

https://gerrit.wikimedia.org/r/739636

Mentioned in SAL (#wikimedia-operations) [2021-11-18T07:35:47Z] <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.9/maintenance/migrateRevisionActorTemp.php: Backport: [[gerrit:739636|maintenance: Add waitForReplication and sleep in migrateRevisionActorTemp (T275246)]] (duration: 01m 04s)

Change 739935 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] maintenance: Add ability to start from rev_id to migrateRevisionActorTemp

https://gerrit.wikimedia.org/r/739935

Change 739935 merged by jenkins-bot:

[mediawiki/core@master] maintenance: Add ability to start from rev_id to migrateRevisionActorTemp

https://gerrit.wikimedia.org/r/739935

Change 740807 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Set test wikis to write both for actor temp table migration

https://gerrit.wikimedia.org/r/740807

Change 740807 merged by jenkins-bot:

[operations/mediawiki-config@master] Set test wikis to write both for actor temp table migration

https://gerrit.wikimedia.org/r/740807

Mentioned in SAL (#wikimedia-operations) [2021-11-23T11:02:21Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:740807|Set test wikis to write both for actor temp table migration (T275246)]] (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2021-11-23T11:08:14Z] <Amir1> start of mwscript migrateRevisionActorTemp.php --wiki=testwiki --sleep=5 (T275246)

Change 741082 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Set actor migration to write both on all wikis

https://gerrit.wikimedia.org/r/741082

Change 741082 merged by jenkins-bot:

[operations/mediawiki-config@master] Set actor migration to write both on all wikis

https://gerrit.wikimedia.org/r/741082

Mentioned in SAL (#wikimedia-operations) [2021-11-24T08:51:31Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:741082|Set actor migration to write both on all wikis (T275246)]] (duration: 00m 57s)

Mentioned in SAL (#wikimedia-operations) [2021-11-24T16:13:30Z] <Amir1> start of "foreachwikiindblist s3 migrateRevisionActorTemp.php --sleep=2" in mwmaint1002 in a screen. It will take a month or so (T275246)