Find and implement a solution to avoid the wrong revisions in the history
Closed, ResolvedPublic8 Story Points

Description

Motivation
When importing a file using Special:ImportFile the version history is imported and there are one or two revisions for the import: the import Imported with FileImporter from… and optionally the change by the user on the page-content while importing.
In between the original history and those two revisions there is another revision that should not be present: This has the timestamp of the import but the author who uploaded the latest file revision and the edit summary <USER> upload bot uploaded File:<Filename>". This revision should not exist.

Example:

Previous work
T192352: Investigation: Fake Revision in version history

Task
Per T192352#4174039 there are two ways we could go:

  • let the fake revision be created and remove it immediately. What would be the implications?
  • change the order of imports: In T147451: Gracefully handle transfer failures we decided not to import any text as long as the file import wasn't finished. What would be the impliecations of changing that?
  • due to findings in T193621#4191044 we will avoid the creation of that revision in the first place [1]

Acceptance Criteria

  • Have a clear idea how the fake revision can be removed, and know about the consequences and costs of it
  • Have it implemented and working

[1] https://phabricator.wikimedia.org/source/mediawiki/browse/master/includes/filerepo/file/LocalFile.php;98e01050c16e9a06cc4dab1a2c3456307a6c04f2$1659

Lea_WMDE created this task.May 2 2018, 11:38 AM
Restricted Application added a project: TCB-Team. · View Herald TranscriptMay 2 2018, 11:38 AM
Tobi_WMDE_SW set the point value for this task to 5.May 2 2018, 2:37 PM
WMDE-Fisch moved this task from Sprint Backlog to Doing on the WMDE-QWERTY-Sprint-2018-05-02 board.

Change 431570 had a related patch set uploaded (by WMDE-Fisch; owner: WMDE-Fisch):
[mediawiki/core@master] Add setting to control the creation of NullRevision on upload

https://gerrit.wikimedia.org/r/431570

Change 431580 had a related patch set uploaded (by WMDE-Fisch; owner: WMDE-Fisch):
[mediawiki/extensions/FileImporter@master] Do not add NullRevision when using ImportableUploadRevisionImporter

https://gerrit.wikimedia.org/r/431580

WMDE-Fisch renamed this task from Find a solution for removing the fake revision in the version history to Find and implement a solution to avoid the wrong revisions in the history.May 8 2018, 3:43 PM
WMDE-Fisch updated the task description. (Show Details)
WMDE-Fisch changed the point value for this task from 5 to 8.

After digging deeper into MW core here and the FileImporter I came to the conclusion ( also with feedback from @Addshore and @thiemowmde ) that:

  • changing the order of the import is not advisable since broken file imports could end up with no page attached
  • removing the revision cannot be done in a good justifiable way
  • avoiding the revision in the first place is the definite way to go here

So the task evolved into being about avoiding that revision in the first place with the implementing patches above.

WMDE-Fisch updated the task description. (Show Details)May 8 2018, 3:50 PM

Change 431570 merged by jenkins-bot:
[mediawiki/core@master] Add setting to control the creation of NullRevision on upload

https://gerrit.wikimedia.org/r/431570

Change 431580 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Do not add NullRevision when using ImportableUploadRevisionImporter

https://gerrit.wikimedia.org/r/431580

thiemowmde moved this task from Review to Demo on the WMDE-QWERTY-Sprint-2018-05-02 board.
thiemowmde triaged this task as Normal priority.

I could not test this, because when trying to open SpecialPage:Import I instead see

Ouch. This needs a MediaWiki core update. The core patch is merged, but the version running on http://mwfileimport.wmflabs.org/wiki/index.php/Special:Version is from 19 April 2018, about a month old. What are the update plans for core on this machine?

Change 433158 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Add backwards compatibility for not yet released core change

https://gerrit.wikimedia.org/r/433158

@thiemowmde yes, this instance should be updated.

@Addshore or @WMDE-Fisch could you add @thiemowmde to the list of maintainers of https://tools.wmflabs.org/openstack-browser/project/Mwfileimport?

Change 433158 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Add backwards compatibility for not yet released core change

https://gerrit.wikimedia.org/r/433158

@thiemowmde @Tobi_WMDE_SW
So the extension master is pulled via cron job every few minutes. Core updates need to be done manually though. This is done now.

I will add Thiemo to the maintainers list just now.

I will add Thiemo to the maintainers list just now.

Done. Can you confirm @thiemowmde ?

Lea_WMDE closed this task as Resolved.May 16 2018, 10:29 AM

I see my name, but I have no idea what to do on this page: https://tools.wmflabs.org/openstack-browser/project/Mwfileimport. An introduction would be lovely.

I see my name, but I have no idea what to do on this page: https://tools.wmflabs.org/openstack-browser/project/Mwfileimport. An introduction would be lovely.

We can look at it tomorrow. In general that's an overview on the project instances and you now should be able to ssh into web01.mwfileimport.eqiad.wmflabs ( where the MediaWiki installation lies ) for example.

Vvjjkkii renamed this task from Find and implement a solution to avoid the wrong revisions in the history to ysdaaaaaaa.Jul 1 2018, 1:12 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii removed WMDE-Fisch as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed the point value for this task.
Vvjjkkii edited subscribers, added: WMDE-Fisch; removed: gerritbot.
CommunityTechBot set the point value for this task to 8.Jul 1 2018, 4:21 PM
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot assigned this task to WMDE-Fisch.
CommunityTechBot lowered the priority of this task from High to Normal.
CommunityTechBot closed this task as Resolved.
CommunityTechBot renamed this task from ysdaaaaaaa to Find and implement a solution to avoid the wrong revisions in the history.
CommunityTechBot edited subscribers, added: gerritbot; removed: WMDE-Fisch.