Page MenuHomePhabricator

New uploads to commons not visible on other wikis
Closed, ResolvedPublic

Event Timeline

Reedy raised the priority of this task from to Needs Triage.
Reedy updated the task description. (Show Details)
Reedy added a project: Multimedia.
Reedy added a subscriber: Reedy.

It's a 1.26wmf5 vs 1.26wmf6 bug

EoRdE6 triaged this task as Unbreak Now! priority.May 20 2015, 2:22 AM
EoRdE6 added a subscriber: EoRdE6.

Adding @greg so that he's aware of this issue.

This issue is pretty bad, as it affects transclusions too, which means affected images are actually unusable. Example: https://en.wikipedia.org/wiki/User:Deskana/sandbox

It's a 1.26wmf5 vs 1.26wmf6 bug

There are three core commits in that range which are obviously file-handling-related:
https://gerrit.wikimedia.org/r/#/c/206968
https://gerrit.wikimedia.org/r/#/c/207004
https://gerrit.wikimedia.org/r/#/c/210091
which are all about multidatacenter-aware caching and seem harmless (even if caching got broken somehow, file lookups would just fall back to querying the DB).

I tried to manually retrace the steps taken when calling wfFindFile:

tgr@terbium:~$ mwscript eval.php --wiki=enwiki
> $t = File::normalizeTitle( 'File:United_States_change_frame_1870-07-15.png' );
> $commons = RepoGroup::singleton()->getRepoByName( 'shared' );
> $f = $commons->newFile( $t );
> $dbr = $commons->getMasterDB();
> echo $dbr->getWikiId();

// $row = $dbr->selectRow( 'image', $this->getCacheFields( 'img_' ), array( 'img_name' => $this->getName() ), $fname );
> echo $dbr->getWikiId();
commonswiki
> echo $dbr->selectSQLText( 'image', $f->getCacheFields( 'img_' ), array( 'img_name' => $f->getName() ), 'eval.php' );
SELECT  img_size,img_width,img_height,img_bits,img_media_type,img_major_mime,img_minor_mime,img_metadata,img_timestamp,img_sha1,img_user,img_user_text,img_description  FROM `image`   WHERE img_name = 'File:United_States_change_frame_1870-07-15.png'

Note the file prefix in the img_name condition; that should not be there.

This is not specific to the wikipedias so not sure if it's causing the current problem or it's a sleeping bug in an unused code path.

Tgr lowered the priority of this task from Unbreak Now! to Medium.May 20 2015, 2:45 PM

Nevermind, user error, File::normalizeTitle does not accept prefixes.

File:United States change frame 1870-07-15.png works now; not sure if it's a side-effect of me messing around in eval.php or some cache just timed out. The examples mentioned in the enwiki discussion still not work. Most new uploads to Commons however work, so decreasing priority.

This is a cache issue; RepoGroup::singleton()->getRepoByName( 'shared' )->findFile( <title> ) fails but RepoGroup::singleton()->getRepoByName( 'shared' )->findFile( <title>, array( 'latest' => true ) ) works (and fixes that specific file for all wikis).

I can reproduce the error by looking at the would-be description page on enwiki first before uploading on commons. File lookups are cached for 24h even if the file does not exist; with the multiple datacenter support cache semantics have changed and set() does not reliable overwrite an already cached value anymore. I'm guessing cache records for non-existing files need a delete() call when the file is uploaded.

I think there should be at least a Commons site notice to warn of this crucial bug ("Priority: normal" must be a joke, right?). It's a really bad user experience to upload stuff and then not be able to use it on the home wikipedia then, without knowing why at all. It took me half an hour to find this bug report...

LocalFile::recordUpload2() does call delete(). I also tried to call it manually and it made no difference. Also, caching seems to work fine (with a few minutes of delay) but the file still does not show up:

tgr@terbium:~$ mwscript eval.php --wiki=enwiki
> $f = RepoGroup::singleton()->getRepoByName( 'shared' )->newFile( 'Tgr_test_12345.png' );
> $key = $f->getCacheKey();
> $cache = ObjectCache::getMainWANInstance();
> $v = $cache->get( $key );
> echo $v['fileExists'];
1

@AndreasPraefcke, it only affects a fraction of new uploads as far as I could see.

@Tgr: Ah, thanks for the info. I affected all three of my uploads that I tried to use on de.wikipedia today, hence I thought it was really very urgent.

MGChecker raised the priority of this task from Medium to High.May 20 2015, 6:59 PM

I think such a bug should have a high priority, because it is an key element of Wikipedia and Wikimedia Commons and I have seen more than a few cases in which using of images from Commons hasn´t worked.

Please stop playing with the priority up and down and let @greg or @Aklapper deal with internal WMF priorities. We all agree it is important and a high severity, the exact priority level is less important.

After today's deploy this is expected to not be a problem.

@aaron for posterity's sake, it would be nice to know WHY after today's deploy it's no longer a problem :)

aaron claimed this task.

@aaron for posterity's sake, it would be nice to know WHY after today's deploy it's no longer a problem :)

One version used the WAN cache while the other used $wgMemc.