Page MenuHomePhabricator

Remove old archive.ar_text/archive.ar_flags
Closed, ResolvedPublic

Description

Bug 31204 (T33204) removes an old field from the user table. There are also old fields in the archive table. Please remove archive.ar_text/archive.ar_flags from the database, because these are from MediaWiki 1.4.

Thanks.

Details

Reference
bz31223

Event Timeline

bzimport raised the priority of this task from to Normal.Nov 21 2014, 11:57 PM
bzimport set Reference to bz31223.
bzimport added a subscriber: Unknown Object (MLST).
brion added a comment.Sep 28 2011, 8:51 PM

These fields will still be in use on production sites; anything deleted during those older installations will still be in the database. (Existing deleted entries are not automatically migrated on access or anything.)

I suppose we could explicitly migrate any remaining ones over to the text table, though.

(In reply to comment #1)

These fields will still be in use on production sites; anything deleted during
those older installations will still be in the database. (Existing deleted
entries are not automatically migrated on access or anything.)
I suppose we could explicitly migrate any remaining ones over to the text
table, though.

I suppose it'd be worth checking on enwiki or something, how many rows that amounts to...

Needs a maintenance script first (bug 34925)

Change 393929 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@master] Drop archive.ar_text and ar_flags

https://gerrit.wikimedia.org/r/393929

cicalese renamed this task from Remove old archive.ar_text/archive.ar_flags to [MCR] Remove old archive.ar_text/archive.ar_flags.Jan 11 2018, 6:17 PM
Krinkle updated the task description. (Show Details)Jan 11 2018, 8:20 PM
Krinkle removed a subscriber: wikibugs-l-list.
CCicalese_WMF renamed this task from [MCR] Remove old archive.ar_text/archive.ar_flags to Remove old archive.ar_text/archive.ar_flags.Jan 12 2018, 4:49 PM
Abit added a subscriber: Abit.Mar 6 2018, 1:51 AM

@Anomie, what is this task blocked by?

Change 393929 merged by jenkins-bot:
[mediawiki/core@master] Drop archive.ar_text and ar_flags

https://gerrit.wikimedia.org/r/393929

hashar added a subscriber: hashar.Apr 27 2018, 9:45 AM

On my local machine using sqlite, update.php seems to be failing due to https://gerrit.wikimedia.org/r/#/c/393929/:

Populating ar_rev_id.
...Update 'PopulateArchiveRevId' already logged as completed.
Making ar_rev_id not nullable ...[c138d1f7c646ee887272a4b9] [no req]   Wikimedia\Rdbms\DBQueryError from line 1453 of /home/hashar/projects/mediawiki/core/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
Query:
INSERT OR IGNORE INTO archive_tmp (
 ar_id, ar_namespace, ar_title, ar_comment, ar_comment_id, ar_user,
 ar_user_text, ar_actor, ar_timestamp, ar_minor_edit, ar_rev_id,
 ar_text_id, ar_deleted, ar_len, ar_page_id, ar_parent_id, ar_sha1,
 ar_content_model, ar_content_format)
 SELECT
 ar_id, ar_namespace, ar_title, ar_comment, ar_comment_id, ar_user,
 ar_user_text, ar_actor, ar_timestamp, ar_minor_edit, ar_rev_id,
 ar_text_id, ar_deleted, ar_len, ar_page_id, ar_parent_id, ar_sha1,
 ar_content_model, ar_content_format
 FROM archive
Function: Wikimedia\Rdbms\Database::sourceFile( /home/hashar/projects/mediawiki/core/maintenance/sqlite/archives/patch-ar_rev_id-not-null.sql )
Error: 1 no such column: ar_actor

I haven't investigated further. Not CI does not run SQLite, though one can trigger a job using SQLite by commenting check experimental in Gerrit which triggers quibble-vendor-sqlite-php70-docker. With a fresh database, the install works just fine so that must be a migration patch fails to check whether ar_actor actually exists?

Reedy added a comment.Apr 27 2018, 2:45 PM

populateArchiveRevId runs patch-ar_rev_id-not-null.sql in 1.31. However, the actor creations/migrations should've already run also, but earlier in the 1.31 updaters?

			// 1.31
			[ 'addTable', 'content', 'patch-content.sql' ],
			[ 'addTable', 'content_models', 'patch-content_models.sql' ],
			[ 'addTable', 'slots', 'patch-slots.sql' ],
			[ 'addField', 'slots', 'slot_origin', 'patch-slot-origin.sql' ],
			[ 'addTable', 'slot_roles', 'patch-slot_roles.sql' ],
			[ 'migrateArchiveText' ],
			[ 'addTable', 'actor', 'patch-actor-table.sql' ],
			[ 'migrateActors' ],
			[ 'modifyField', 'revision', 'rev_text_id', 'patch-rev_text_id-default.sql' ],
			[ 'modifyTable', 'site_stats', 'patch-site_stats-modify.sql' ],
			[ 'populateArchiveRevId' ],
			[ 'addIndex', 'recentchanges', 'rc_namespace_title_timestamp',
				'patch-recentchanges-nttindex.sql' ],

Not CI does not run SQLite,

But the standard jobs do run updates for SQLite, see https://integration.wikimedia.org/ci/job/mediawiki-phpunit-hhvm-jessie/21761/console for example where a build failed because of out-of-order updates.

As Reedy noted, the ar_actor column should have been added by patch-actor-table.sql, run a few lines before populateArchiveRevId() tries to execute patch-ar_rev_id-not-null.sql.

Change 430638 had a related patch set uploaded (by Jforrester; owner: Anomie):
[mediawiki/core@REL1_31] Drop archive.ar_text and ar_flags

https://gerrit.wikimedia.org/r/430638

Change 430638 merged by jenkins-bot:
[mediawiki/core@REL1_31] Drop archive.ar_text and ar_flags

https://gerrit.wikimedia.org/r/430638

hashar added a comment.EditedMay 4 2018, 7:51 AM

Not CI does not run SQLite,

But the standard jobs do run updates for SQLite, see https://integration.wikimedia.org/ci/job/mediawiki-phpunit-hhvm-jessie/21761/console for example where a build failed because of out-of-order updates.
As Reedy noted, the ar_actor column should have been added by patch-actor-table.sql, run a few lines before populateArchiveRevId() tries to execute patch-ar_rev_id-not-null.sql.

Sorry I got confused about SQLite / MySQL ...

Looks like somehow archives/patch-actor-table.sql never had a chance to run completely. It creates the actor table and add the column ar_actor. Update.php does state ...actor table already exists.. So I guess it is a local issue of some sort.

I got it sorted out by manually creating archive.ar_actor. So feel free to close it is probably a off by one error on my local machine.

Anomie added a comment.May 4 2018, 3:25 PM

That's a hazard of the way we do schema changes and the fact that MySQL doesn't allow DDL to be done in transactions: a patch file can wind up being partially done if it errors out in the middle, and depending on the specific SQL statements it may or may not be detected and may or may not actually run the next time.

There are a few possible workarounds:

  • Change the patch file to create the table last.
  • Split the patch file into multiple patch files.
  • Just wait for T191231 and plan things better there.
Anomie added a comment.May 4 2018, 3:26 PM

Or, I suppose, you might have accidentally dropped the column at some point, e.g. when running update.php for some pre-actor-table patch that modified archive after having run update.php for the actor table patch.

hashar closed this task as Resolved.May 4 2018, 7:55 PM

T191231 looks like the nice long term solution. Happy to see someone thought about that.

Lets mark this resolve. It is most definitely a one off issue on my machine. Thanks @Anomie and @Reedy !