Page MenuHomePhabricator
Create Task
Maniphest T183474

Deleting a file from its original location AFTER moving to commons can result in the File page on commons containing incorrect links in its 'Global file usage' section
Closed, ResolvedPublic
Actions

Description

If a user deletes the file from its original location AFTER moving it to commons, then sometimes the File page on commons will contain incorrect links in its 'Global file usage' section

See for example https://commons.wikimedia.org/wiki/File:FiveThirtyEight_Logo.svg

The link to the non-existent page is displayed in the 'Global usage' section because there is an entry in the globalimagelinks table for it - see the third line of the resultset here https://quarry.wmflabs.org/query/23571

... so it looks like this particular bug could be caused by EITHER

  • an error writing to the globalimagelinks table in the GlobalUsage extension (I've been through the code here, and can't see how it'd be possible to add a link to a non-existent page, but the code is new to me and I could easily be missing something)
  • an error writing to the globalimagelinks table when transferring the file data from its original location to commons ... @Magog_the_Ogre I don't know how you copied the data across, but perhaps you used a tool that might have a bug in it?

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
Copy file usage from the correct database when local file is deletedBartosz Dziewońskimediawiki/extensions/GlobalUsagemaster+4 -3
Customize query in gerrit

Related Objects
Search...

Event Timeline

Cparle created this task.Dec 21 2017, 4:13 PM
Cparle triaged this task as Medium priority.
Cparle updated the task description. (Show Details)Dec 21 2017, 4:53 PM
• Ramsey-WMF moved this task from Untriaged to Tracking on the Multimedia board.Dec 22 2017, 5:20 PM
Magog_the_Ogre mentioned this in T183478: Deleting a file from a wiki AFTER moving to commons: pages on the original wiki linking to the file may have broken links until the page is purged.Feb 16 2018, 4:21 AM
Magog_the_Ogre mentioned this in T162532: Deleting a file which is on the shared repository doesn't properly update all the local links.
matmarex added a project: GlobalUsage.Oct 31 2022, 2:52 AM
matmarex merged a task: T218778: GlobalUsage duplicates Commons usage to other wikis' sections with broken links.Oct 31 2022, 3:15 AM
matmarex added subscribers: AntiCompositeNumber, Geagea, eranroz, matmarex.
matmarex added a comment.Oct 31 2022, 3:32 AM

I've been looking into this issue on T218778, just for fun, because it seemed so implausible. (And also T199398, which is a different problem with local-global file usage.)

With the information in this task, I can see why it happens: it's caused by this code:

Its purpose is to copy the local file links to global file links when a local file is deleted but global file exists (for example, after the file was imported to Commons).

Unfortunately, it's querying the local links from the wrong database – $this->dbr refers to the database containing the globalimagelinks table, which in our configuration is the commonswiki database. Therefore Commons' local file links are copied with the wrong wiki ID (and the correct wiki's local file links are not recorded as global at all).

matmarex added a comment.Oct 31 2022, 3:48 PM

(This has been broken since 2016: 3e922e17d53dbd6b771796c125c46118bb9c3b46)

gerritbot added a comment.Oct 31 2022, 3:48 PM

Change 851098 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/GlobalUsage@master] Copy file usage from the correct database when local file is deleted

https://gerrit.wikimedia.org/r/851098

gerritbot added a project: Patch-For-Review.Oct 31 2022, 3:48 PM
matmarex added a comment.Oct 31 2022, 7:34 PM

I was curious how many of the existing entries are affected by this bug:

select count(*)
from globalimagelinks
inner join page on gil_page=page_id and gil_page_namespace_id=page_namespace and gil_page_title=page_title
where gil_wiki != 'commonswiki'

This gives 1634414 rows, which is about 0.25% of the 641608450 rows in the globalimagelinks table. (There might be a small number of false positives in this query, see T218778#8355511.) They could be fixed with the refreshGlobalimagelinks.php maintenance script.

matmarex mentioned this in T322588: Run `refreshGlobalimagelinks.php --pages=nonexisting` from the GlobalUsage extension.Nov 7 2022, 10:31 PM
gerritbot added a comment.Nov 25 2022, 6:46 PM

Change 851098 merged by jenkins-bot:

[mediawiki/extensions/GlobalUsage@master] Copy file usage from the correct database when local file is deleted

https://gerrit.wikimedia.org/r/851098

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.12; 2022-11-28).Nov 25 2022, 7:00 PM
Maintenance_bot removed a project: Patch-For-Review.Nov 25 2022, 7:30 PM
matmarex closed this task as Resolved.Nov 26 2022, 1:49 PM
matmarex claimed this task.

This will stop new cases of the problem from occurring.

To resolve existing cases, we'll need to run a maintenance script, I filed a separate task about it: T322588: Run `refreshGlobalimagelinks.php --pages=nonexisting` from the GlobalUsage extension. I'm planning to do that next week after the code is deployed.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits