Deploy refactored comment storage
Open, LowPublic

Description

The high-level checklist:

  • 0. Set the configuration flag to "old" (see T166732) (not needed, default is now "old")
  • 1. Merge the first patch for T166732: Refactor comment storage in the database and abstract access in MediaWiki (adding new schemas and code)
  • 2. Perform schema change (T174569)
  • 3. Turn the feature flag to "read+write both". See if stuff breaks.
  • 3.1. Announce the pending change to wikitech-l@ and cloud@, and give time for people to update.
  • 3.2. Make sure all deployed extensions are updated.
  • 4. Turn the feature flag to "write new only, read both". See if stuff breaks.
  • 5. Run the maintenance script(s) to migrate all the old stuff to new stuff, blanking the old stuff in the process.
  • 6. Turn the feature flag to "new only".
  • 7. Merge the second patch for T166732 (removing old schemas and code)
    • 7.1. Submit schema change (task TBD)

We can easily do steps 2–6 for individual wikis rather than rolling it out all at once. I'm filing this task now mainly so that possibility can be discussed: do we want to do that, and if so decide which wikis to do preliminary deploys to.

There are a very large number of changes, so older changes are hidden. Show Older Changes
Anomie updated the task description. (Show Details)Sep 7 2017, 2:38 PM
bd808 updated the task description. (Show Details)Nov 3 2017, 8:35 PM
bd808 added a subscriber: bd808.

Change 392082 had a related patch set uploaded (by Anomie; owner: Anomie):
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on Beta Cluster

https://gerrit.wikimedia.org/r/392082

Change 392082 merged by jenkins-bot:
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on Beta Cluster

https://gerrit.wikimedia.org/r/392082

If someone has time, please give an update to https://meta.wikimedia.org/wiki/Community_Tech/Edit_summary_length_for_non-Latin_languages I could do it myself, but I cannot find what is the future character limit?

Regarding time, this has suffered a delay- because of the incident on s5, which delayed s8 deployment, which by itself blocks this deployment means we will not be ready by December. Ops/releng advised not to deploy large changes on December, so we will probably be able to continue (?) except on s5, which put us on mid-January for the schema change.

I'll update it.

Change 395073 had a related patch set uploaded (by Anomie; owner: Anomie):
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on test wikis

https://gerrit.wikimedia.org/r/395073

Change 395073 merged by jenkins-bot:
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on test wikis

https://gerrit.wikimedia.org/r/395073

Mentioned in SAL (#wikimedia-operations) [2017-12-04T19:31:55Z] <legoktm@tin> Synchronized wmf-config/InitialiseSettings.php: Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on test wikis (T166733) (duration: 00m 43s)

Krinkle added a subscriber: Krinkle.EditedDec 4 2017, 11:28 PM

Change 395073 merged by jenkins-bot:
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on test wikis

https://gerrit.wikimedia.org/r/395073

This was reverted in git by https://gerrit.wikimedia.org/r/#/c/395077/, which is logged as having been deployed:

20:07 anomie@tin: Synchronized wmf-config/InitialiseSettings.php: Revert wgCommentTableSchemaMigrationStage change, breaks too much stuff (duration: 00m 43s)

However, tin and terbium are still in the pre-revert state (the revert wasn't even fetched down yet). Which suggests the above scap sync-file was probably a no-op.

Given it's been three hours and I don't want to be one deploying it now, I've re-reverted the commit in git so that the repo reflects actual deployed state.

EDIT: This has now been re-re-reverted and deployed. Thanks!

Anomie added a comment.Dec 5 2017, 2:55 PM

My fault, I forgot the "git pull" before syncing the file out.

Krinkle removed a subscriber: Krinkle.

Change 408536 had a related patch set uploaded (by Anomie; owner: Anomie):
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on test wikis

https://gerrit.wikimedia.org/r/408536

Change 408536 merged by jenkins-bot:
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on test wikis

https://gerrit.wikimedia.org/r/408536

Anomie updated the task description. (Show Details)Feb 7 2018, 5:06 PM

Change 409048 had a related patch set uploaded (by Anomie; owner: Anomie):
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on meta, mw.org

https://gerrit.wikimedia.org/r/409048

Change 409048 merged by jenkins-bot:
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on meta, mw.org

https://gerrit.wikimedia.org/r/409048

It may be time to mention this in Tech News. Perhaps something like this

The way edit summaries and log comments are stored in the database is being changed. In the future this will [[phab:T6714|allow for longer comments]].

Note that the web UI is not yet updated to allow entry of longer comments. The task for doing that is T185948: Update web UI for increased comment length.

It may be time to mention this in Tech News. Perhaps something like this

The way edit summaries and log comments are stored in the database is being changed. In the future this will [[phab:T6714|allow for longer comments]].

Note that the web UI is not yet updated to allow entry of longer comments. The task for doing that is T185948: Update web UI for increased comment length.

I don't think you'll get much useful feedback now. Much better to wait for the UI being updated first and then ask. (You'll probably only get one shot.)

The only feedback I'm really looking for is "oh crap something broke".

Small question regarding old comment format, while BOTH, is the old format truncated in advance, or do you let the database do it on its own?

Anomie added a comment.Feb 9 2018, 3:37 PM

It's truncated in advance. In case you're curious, the specific place that happens is CommentStore.php line 533.

Change 410197 had a related patch set uploaded (by Anomie; owner: Anomie):
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on group 0

https://gerrit.wikimedia.org/r/410197

Change 410197 merged by jenkins-bot:
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on group 0

https://gerrit.wikimedia.org/r/410197

Change 412955 had a related patch set uploaded (by Anomie; owner: Anomie):
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on group 1

https://gerrit.wikimedia.org/r/412955

Change 412955 merged by jenkins-bot:
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH on group 1

https://gerrit.wikimedia.org/r/412955

Change 415043 had a related patch set uploaded (by Anomie; owner: Anomie):
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH everywhere

https://gerrit.wikimedia.org/r/415043

Change 415043 merged by jenkins-bot:
[operations/mediawiki-config@master] Set wgCommentTableSchemaMigrationStage = MIGRATION_WRITE_BOTH everywhere

https://gerrit.wikimedia.org/r/415043

Anomie updated the task description. (Show Details)
Nirmos added a subscriber: Nirmos.Apr 19 2018, 3:54 PM
Anomie updated the task description. (Show Details)Mon, Sep 17, 3:04 PM

Change 460916 had a related patch set uploaded (by Anomie; owner: Anomie):
[operations/mediawiki-config@master] Set actor and comment schema migrations for Beta Cluster

https://gerrit.wikimedia.org/r/460916

Change 460916 merged by jenkins-bot:
[operations/mediawiki-config@master] Set actor and comment schema migrations for Beta Cluster

https://gerrit.wikimedia.org/r/460916

Change 460924 had a related patch set uploaded (by Anomie; owner: Anomie):
[operations/mediawiki-config@master] Set CommentTableSchemaMigrationStage => WRITE_BOTH on test wikis, mw.org

https://gerrit.wikimedia.org/r/460924

Change 460924 merged by jenkins-bot:
[operations/mediawiki-config@master] Set CommentTableSchemaMigrationStage => WRITE_NEW on test wikis, mw.org

https://gerrit.wikimedia.org/r/460924

Mentioned in SAL (#wikimedia-operations) [2018-09-17T16:36:18Z] <anomie@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Setting wgCommentTableSchemaMigrationStage = WRITE_NEW on test wikis, mw.org (T166733) (duration: 00m 50s)