Page MenuHomePhabricator

Drop filejournal table from WMF
Open, LowPublic

Description

Schema Change:

All wikis
tables: filejournal
changes: Drop fj_new_sha1 field
commit status: Unknown

Progress

  • s1
  • s2
  • s3
  • s4
  • s5
  • s6
  • s7
  • s8
  • labswiki doesn't have this table
  • labtestwiki

Details

Reference
bz49195

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 1:52 AM
bzimport set Reference to bz49195.
greg created this task.Jun 5 2013, 5:12 PM

Definitely drop this?

The field is still referenced at least once in core:

$ grep -nr fj_new_sha1 *
includes/filebackend/filejournal/DBFileJournal.php:71

greg added a comment.Jul 29 2013, 7:57 PM

CC'ing Asher as he's the one who added it:
https://wikitech.wikimedia.org/w/index.php?title=Schema_changes&diff=prev&oldid=44661

(/me just did a manual binary search on that page's history, ugh)

It looks like this is a core table, but it's not documented at https://www.mediawiki.org/wiki/Manual:Database_layout ...

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 2 2015, 4:39 PM
Krenair removed Springle as the assignee of this task.Oct 4 2015, 2:52 AM
Reedy changed the task status from Open to Stalled.Apr 26 2017, 2:35 PM
Reedy removed a subscriber: Springle.
Restricted Application added projects: Multimedia, Commons. · View Herald TranscriptApr 26 2017, 2:35 PM
Restricted Application added a project: Multimedia. · View Herald TranscriptDec 8 2017, 7:43 PM
Marostegui moved this task from Triage to Backlog on the DBA board.Jul 27 2018, 4:00 PM
greg removed a subscriber: greg.Nov 22 2018, 8:36 AM
Marostegui edited projects, added MediaWiki-Rdbms; removed DBA.Nov 23 2018, 6:40 AM
Marostegui added subscribers: Banyek, Reedy, Marostegui.

This needs to be removed from code first:

includes/filebackend/filejournal/DBFileJournal.php:				'fj_new_sha1' => $entry['newSha1'],
maintenance/archives/patch-filejournal.sql:  fj_new_sha1 varbinary(32) NOT NULL default '',

I am removing the DBA tag until this is merged and good to go. I will remain subscribed to the task to be aware of its progress.

Krinkle renamed this task from Drop fj_new_sha1 field to Drop filejournal.fj_new_sha1 field.Jul 18 2019, 9:47 PM
Krinkle renamed this task from Drop filejournal.fj_new_sha1 field to Drop filejournal.fj_new_sha1 field from core and WMF.
Krinkle removed a project: MediaWiki-Rdbms.
aaron added a subscriber: aaron.Jul 18 2019, 9:51 PM

Dropping the field doesn't make sense, but dropping the whole table does. We do not use that class in production (and it is optional within MW core).

Krinkle renamed this task from Drop filejournal.fj_new_sha1 field from core and WMF to Drop filejournal table from core and WMF.Jul 18 2019, 9:53 PM
Krinkle renamed this task from Drop filejournal table from core and WMF to Drop filejournal table from WMF.
Krinkle changed the task status from Stalled to Open.
Krinkle updated the task description. (Show Details)
Krinkle added a project: DBA.
Marostegui updated the task description. (Show Details)

Removing the DBA tag per T51195#4769106ç
The table is still present on the archives/patch-filejournal.sql

cat archives/patch-filejournal.sql
-- File backend operation journal
CREATE TABLE /*_*/filejournal (
  -- Unique ID for each file operation
  fj_id bigint unsigned NOT NULL PRIMARY KEY auto_increment,
  -- UUID of the batch this operation belongs to
  fj_batch_uuid varbinary(32) NOT NULL,
  -- The registered file backend name
  fj_backend varchar(255) NOT NULL,
  -- The storage path that was affected (may be internal paths)
  fj_path blob NOT NULL,
  -- Primitive operation description (create/update/delete)
  fj_op varchar(16) NOT NULL default '',
  -- SHA-1 file content hash in base-36
  fj_new_sha1 varbinary(32) NOT NULL default '',
  -- Timestamp of the batch operation
  fj_timestamp varbinary(14) NOT NULL default ''
) /*$wgDBTableOptions*/;

CREATE INDEX /*i*/fj_batch_id ON /*_*/filejournal (fj_batch_uuid);
CREATE INDEX /*i*/fj_timestamp ON /*_*/filejournal (fj_timestamp);
aaron added a comment.Mon, Aug 12, 6:26 PM

It's an optional table, not installed by update.php.

Maybe those can go in a different directory for clarity.

Marostegui added a project: DBA.

Ah! Ok :)
Didn't know
Thanls for the clarification

@aaron are you sure this table isn't created by default? I just checked the most recent wiki we created yuewiktionary T205714: Prepare and check storage layer for yuewiktionary which was created the 14th Nov 2018 and filejournal table is there.

-rw-rw---- 1 mysql mysql 128K Nov 14  2018 filejournal.ibd

I have checked because I don't want to delete this table everywhere if it is still going to be created upon new wikis creations.

Marostegui moved this task from Backlog to Next on the DBA board.Tue, Aug 13, 11:11 AM

Confirmed with the wiki that was created past Wed 14th Aug 2019 (T210762#5413963) - this table is still created when a new wiki is created:

root@db1075.eqiad.wmnet[napwikisource]> show tables like 'filej%';
+----------------------------------+
| Tables_in_napwikisource (filej%) |
+----------------------------------+
| filejournal                      |
+----------------------------------+
1 row in set (0.00 sec)

Moving this task to our Blocked column until this is sorted and remove from any code reference, as I want to avoid the situation where we delete it now, and it still gets created on every new wiki.

Change 530847 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/extensions/WikimediaMaintenance@master] Stop adding filejournal table to new wikis

https://gerrit.wikimedia.org/r/530847

Change 530847 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMaintenance@master] Stop adding filejournal table to new wikis

https://gerrit.wikimedia.org/r/530847

Thanks for getting that merged @Reedy - this can now proceed then!