Page MenuHomePhabricator

Allow write old and new for event table migration
Closed, ResolvedPublic

Description

It should be possible to write new and old for a period while the migration is being completed.

Update code to support write both in:

  • The many methods that insert, delete and change cu_changes in Hooks.php
  • PopulateCheckUserTable maintenance script
  • MoveLogEntriesFromCuChanges maintenance script

Related Objects

StatusSubtypeAssignedTask
ResolvedSecurityZabe
ResolvedSecurityDreamy_Jazz
ResolvedSecurityDreamy_Jazz
ResolvedSecurityDreamy_Jazz
OpenBUG REPORTNone
OpenBUG REPORTNone
OpenBUG REPORTNone
OpenDreamy_Jazz
OpenNone
OpenNone
OpenNone
OpenFeatureDreamy_Jazz
DuplicateNone
OpenFeatureNone
OpenFeatureNone
OpenFeatureNone
ResolvedFeatureDreamy_Jazz
ResolvedFeatureDreamy_Jazz
ResolvedFeatureDreamy_Jazz
OpenFeatureNone
OpenFeatureNone
ResolvedBUG REPORTDreamy_Jazz
OpenFeatureNone
OpenFeatureNone
ResolvedGlaisher
OpenNone
OpenFeatureNone
ResolvedGlaisher
ResolvedNiharika
ResolvedNone
ResolvedFeatureDreamy_Jazz
DeclinedNone
ResolvedFeatureDreamy_Jazz
ResolvedFeatureDreamy_Jazz
DuplicateNone
OpenNone
DuplicateNone
ResolvedDreamy_Jazz
ResolvedDreamy_Jazz
ResolvedDreamy_Jazz
OpenFeatureNone
ResolvedDreamy_Jazz
OpenBUG REPORTNone
ResolvedBUG REPORTDreamy_Jazz
OpenNone
OpenFeatureNone
DeclinedFeatureNone
OpenFeatureNone
OpenDreamy_Jazz
In Progresskostajh
OpenNone
OpenNone
Resolvedtstarling
OpenNone
OpenTchanders
OpenFeatureDreamy_Jazz
OpenNone
ResolvedDreamy_Jazz
OpenDreamy_Jazz
ResolvedDreamy_Jazz
ResolvedBUG REPORTDreamy_Jazz
ResolvedDreamy_Jazz
OpenDreamy_Jazz
ResolvedDreamy_Jazz
ResolvedDreamy_Jazz
ResolvedDreamy_Jazz
ResolvedBUG REPORTDreamy_Jazz
ResolvedBUG REPORTDreamy_Jazz
OpenDreamy_Jazz
ResolvedDreamy_Jazz

Event Timeline

Change 886482 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Add support for writing both new and old for event table migration

https://gerrit.wikimedia.org/r/886482

Change 886482 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Add support for writing both new and old for event table migration

https://gerrit.wikimedia.org/r/886482

Change 937171 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Add support to write new and old to MoveLogEntriesFromCuChanges

https://gerrit.wikimedia.org/r/937171

Change 937174 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Add support for write new and old to populateCheckUserTable.php

https://gerrit.wikimedia.org/r/937174

Change 939289 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Follow-up: Add support for writing both new and old to Hooks.php

https://gerrit.wikimedia.org/r/939289

Change 886482 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Add support for writing both new and old to Hooks.php

https://gerrit.wikimedia.org/r/886482

Change 939289 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Follow-up: Add support for writing both new and old to Hooks.php

https://gerrit.wikimedia.org/r/939289

Change 937171 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Add support to write new and old to MoveLogEntriesFromCuChanges

https://gerrit.wikimedia.org/r/937171

Change 937174 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Add support for write new and old to populateCheckUserTable.php

https://gerrit.wikimedia.org/r/937174

Suggested QA steps are below. Optional steps are those which requre email to be configured on the wiki and thus can be skipped if email is not set up if desired.

For the change named "Add support for writing both new and old to Hooks.php":

  1. Set wgCheckUserEventTablesMigrationStage to SCHEMA_COMPAT_OLD | SCHEMA_COMPAT_WRITE_NEW (this is the default value on the main branch)
  2. Set wgCheckUserLogLogins to true
  3. Note the current time
  4. Login to an account
  5. Make a test edit
  6. Move a page (or perform any other action that causes a log entry to appear in Special:RecentChanges)
  7. (Optional) Visit Special:Email and send an email
  8. Logout of the account
  9. (Optional) Visit Special:PasswordReset and submit the form for a user
  10. Attempt to login as a user with the wrong password
  11. Create an account
  12. Check that rows exist in cu_changes, cu_log_event and cu_private_event where the timestamp is after the timestamp noted at the start of the test.
  13. Check that for each action taken in steps 4 to 11 (excluding those which were skipped) a row exists in cu_changes.
    1. (Optional) Ideally, also match the action to the row - the "actiontext" column should usually indicate what action created the row
    2. Check that these rows, except for the edit, have the column cuc_only_for_read_old set to 1.
  14. Check that for actions taken in steps 6 and 11, a row exists for it in the cu_log_event table - To check this, notate the log ID for the move and account creation and check the IDs are present in the column cule_log_id
  15. Check that for the actions taken in steps 4, 7, 8, 9 and 10 a row exists for this action in the table cu_private_event - Checking what action is associated with the given row in the table can be done by looking at the cupe_action column.

For the change named "Add support to write new and old to MoveLogEntriesFromCuChanges":

  1. Set wgCheckUserEventTablesMigrationStage to SCHEMA_COMPAT_OLD
  2. Set wgCheckUserLogLogins to true
  3. Note the current timestamp
  4. Login to an account
  5. Make a test edit
  6. Move a page
  7. Check that rows only exist in the table cu_changes (i.e. none in cu_log_event or cu_private_event).
    1. Check the rows in cu_changes made after the timestamp noted in step 3 and verify no rows have cuc_only_for_read_old equal to 1.
  8. Set wgCheckUserEventTablesMigrationStage to SCHEMA_COMPAT_OLD | SCHEMA_COMPAT_WRITE_NEW (this is the default in the main branch)
  9. Run the maintenance script maintenance/moveLogEntriesFromCuChanges.php (run with --force as a command line argument if the maintenance script does not run because it has already been run).
  10. Verify that the rows in cu_changes associated with the actions performed in step 3 and 5 have the column cuc_only_for_read_old equal to 1.
  11. Verify that the edit has it's cuc_only_for_read_old equal to 0 (which means no change after running the script).
  12. Verify that no rows exist in cu_log_event that have a timestamp after the timestamp noted in step 3
  13. Verify that two rows exist in cu_private_event with a timestamp after the timestamp noted in step 3
    1. Check that both have the value migrated-cu_changes-log-event for the column cupe_action.

For the patch named "Add support for write new and old to populateCheckUserTable.php":

  1. Remove CheckUser from the list of loaded extensions (i.e. remove from wfLoadExtensions)
  2. Empty the tables cu_changes, cu_log_event and cu_private_event for the test.
  3. Set wgCheckUserEventTablesMigrationStage to SCHEMA_COMPAT_OLD | SCHEMA_COMPAT_WRITE_NEW (this is the default value on the main branch)
  4. Run the maintenance script maintenance/populateCheckUserTable.php with the command line option --force
  5. Check that cu_private_event remained empty
  6. Verify that cu_log_event and cu_changes have been filled with entries

Change 941414 had a related patch set uploaded (by Urbanecm; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@wmf/1.41.0-wmf.18] Add support for writing both new and old to Hooks.php

https://gerrit.wikimedia.org/r/941414

Change 941400 had a related patch set uploaded (by Urbanecm; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@wmf/1.41.0-wmf.18] Follow-up: Add support for writing both new and old to Hooks.php

https://gerrit.wikimedia.org/r/941400

Change 941414 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@wmf/1.41.0-wmf.18] Add support for writing both new and old to Hooks.php

https://gerrit.wikimedia.org/r/941414

Change 941400 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@wmf/1.41.0-wmf.18] Follow-up: Add support for writing both new and old to Hooks.php

https://gerrit.wikimedia.org/r/941400

Mentioned in SAL (#wikimedia-operations) [2023-07-25T13:30:52Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:941414|Add support for writing both new and old to Hooks.php (T341934 T341586)]], [[gerrit:941400|Follow-up: Add support for writing both new and old to Hooks.php (T341586)]]

Mentioned in SAL (#wikimedia-operations) [2023-07-25T13:38:21Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:941414|Add support for writing both new and old to Hooks.php (T341934 T341586)]], [[gerrit:941400|Follow-up: Add support for writing both new and old to Hooks.php (T341586)]] (duration: 07m 28s)

@Dreamy_Jazz Thank you so much for the steps, very helpful! I came across everything as expected as seen in the screenshots. The task will be marked as Done, thanks!

For the change named "Add support for writing both new and old to Hooks.php":

All information

T341586_CheckUser_ClientHintData_OldNew_TableMigration_Hook.png (1×3 px, 739 KB)

For the change named "Add support to write new and old to MoveLogEntriesFromCuChanges":

T341586_CheckUser_ClientHintData_OldNew_TableMigration_Movelog.png (850×2 px, 297 KB)

"Add support for write new and old to populateCheckUserTable.php"

T341586_CheckUser_ClientHintData_OldNew_TableMigration_PopulateCUTable1.png (362×1 px, 71 KB)
T341586_CheckUser_ClientHintData_OldNew_TableMigration_PopulateCUTable2.png (221×409 px, 26 KB)
T341586_CheckUser_ClientHintData_OldNew_TableMigration_PopulateCUTable3.png (225×616 px, 48 KB)