Page MenuHomePhabricator

FileImporter should add entries to the import log
Closed, ResolvedPublic8 Story Points

Description

When the FileImporter successfully finishes an import it should not only add a NullRevision to inform about the import step in the revision history but should also add an entry to the upload log and one to the import log. These should all be associated with the importing user and be timestamped with the import date.

Having the NullRevision and the upload log entry is the default for uploads [1] [2] (although it leads to "doubled" entries on the history page). Since we, apart from the upload, do an import, having an import log entry is the right thing to do.

[1] https://commons.wikimedia.org/w/index.php?title=File:Usenet_Services_Mind_Map.svg&action=history
[2] https://commons.wikimedia.org/w/index.php?title=Special:Log&page=File%3AUsenet+Services+Mind+Map.svg

Acceptance Criteria

  • Add an entry to the import log associated with the null revision of the import

Notes
Most probably we will still just have one entry (the null revision) in the page history.

Dev notes
This includes tests

Details

Related Gerrit Patches:
mediawiki/extensions/FileImporter : masterIntegration test for import log entry
mediawiki/extensions/FileImporter : masterWhen importing a file add an import log entry.
mediawiki/extensions/FileImporter : masterAvoid using the real User class in tests, where possible

Event Timeline

thiemowmde triaged this task as Medium priority.Nov 20 2018, 1:27 PM
thiemowmde moved this task from Incoming to Move files to Commons on the TCB-Team board.
WMDE-Fisch added a comment.EditedNov 23 2018, 12:55 PM

Just as first comment here, when working on this it might help to have a look at LocalFile::recordUpload2()and ImportReporter::reportPage in core.

WMDE-Fisch renamed this task from [PLACEHOLDER] FileImporter logging to FileImporter should add entries to the import and upload log.Nov 23 2018, 4:08 PM
WMDE-Fisch updated the task description. (Show Details)

Change 476270 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[mediawiki/extensions/FileImporter@master] When importing a file add an import log entry.

https://gerrit.wikimedia.org/r/476270

Lea_WMDE updated the task description. (Show Details)Nov 28 2018, 4:52 PM
Lea_WMDE set the point value for this task to 8.
Lea_WMDE moved this task from Backlog to Tickets in sprint on the Move-Files-To-Commons board.
Pikne added a subscriber: Pikne.EditedNov 29 2018, 7:55 AM

Is new type of import that would allow filtering imports where file versions were imported along with page revisions in import log also planned?

As for handling upload log entries, I note that currently only the latest upload log entry is copied from (or generated after) source wiki if there were reuploads, e.g. compare this and this. I suppose that if upload log entries are copied then it'd be consistent to copy all of them along with all file versions that are imported.

Change 476523 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Avoid using the real User class in tests, where possible

https://gerrit.wikimedia.org/r/476523

WMDE-Fisch added a subscriber: Andrew-WMDE.

Change 476523 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Avoid using the real User class in tests, where possible

https://gerrit.wikimedia.org/r/476523

Is new type of import that would allow filtering imports where file versions were imported along with page revisions in import log also planned?
As for handling upload log entries, I note that currently only the latest upload log entry is copied from (or generated after) source wiki if there were reuploads, e.g. compare this and this. I suppose that if upload log entries are copied then it'd be consistent to copy all of them along with all file versions that are imported.

Interesting point, I assumed that this already working. - I created a separate ticket for that T210755

Change 476270 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] When importing a file add an import log entry.

https://gerrit.wikimedia.org/r/476270

Change 476851 had a related patch set uploaded (by WMDE-Fisch; owner: WMDE-Fisch):
[mediawiki/extensions/FileImporter@master] Integration test for import log entry

https://gerrit.wikimedia.org/r/476851

WMDE-Fisch updated the task description. (Show Details)Dec 3 2018, 11:21 AM

Is new type of import that would allow filtering imports where file versions were imported along with page revisions in import log also planned?

We are going to use "transwiki import" as sub type in the import log. This type already exists. Would it make more sense for you if there was another specific type? We thought that this is pretty much a transwiki activity :)

Pikne added a comment.EditedDec 4 2018, 7:29 AM

We are going to use "transwiki import" as sub type in the import log. This type already exists. Would it make more sense for you if there was another specific type? We thought that this is pretty much a transwiki activity :)

I thought that new subtype would be useful as it'd allow filtering file imports. But indeed, then you'd probably have to rename "transwiki import" to make distinction between imports with and without file versions clear. So I see that adding a change tag (T198324) for this purpose might be less complicated and sufficient as well (though it's less convenient to retrieve revisions older than 30 days not present in recent changes).

Lea_WMDE renamed this task from FileImporter should add entries to the import and upload log to FileImporter should add entries to the import log.Dec 4 2018, 1:17 PM
Lea_WMDE updated the task description. (Show Details)
Lea_WMDE moved this task from Review to Demo on the WMDE-QWERTY-Sprint-2018-11-20 board.

Change 476851 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Integration test for import log entry

https://gerrit.wikimedia.org/r/476851

WMDE-Fisch closed this task as Resolved.Jan 17 2019, 4:54 PM
WMDE-Fisch updated the task description. (Show Details)