By the way, I get the same problem with: https://releases.wikimedia.org/mediawiki/1.26/mediawiki-1.26.2.tar.gz. Not with older releases though.

This is nothing to do with Differential, Diffusion, Audit or Phabricator

confirmed the trusty file works:

https://releases.wikimedia.org/debian/dists/trusty-mediawiki/InRelease

jessie file does not:

https://releases.wikimedia.org/debian/dists/jessie-mediawiki/InRelease

but permissions in file system look identical

4.0K -rw-rw-r-- 1 5038 1004 2.8K Sep 2 2015 InRelease

vs.

4.0K -rw-rw-r-- 1 5038 1004 2.8K Sep 17 2015 InRelease

don't see error in apache log either on the backend (bromine). i think we must check on misc-web now

P.S. still no reason that the files are owned by "5038"

I wasn't aware we had a Debian repository outside of apt.wikimedia.org for internal use. Looking at the files there is solely some Parsoid package from September 2015:

https://releases.wikimedia.org/debian/pool/main/p/parsoid/

That is apparently from @GWicke .

Maybe some rsync from labs or a local machine ended up syncing with bad file permissions? I am not sure how maintained those parsoid packages are nor whether they are actively published. Maybe we can consider dropping the whole hierarchy entirely but I digress.

@Ronarts12 for https://releases.wikimedia.org/mediawiki/1.26/mediawiki-1.26.2.tar.gz we have/had an ongoing issue with our frontend cache :-/

on bromine, both files look exactly the same with same weird uid/gid:

$ ls -l /srv/org/wikimedia/releases/debian/dists/{trusty,jessie}-mediawiki/InRelease
-rw-rw-r-- 1 5038 1004 2840 Sep 17  2015 /srv/org/wikimedia/releases/debian/dists/jessie-mediawiki/InRelease
-rw-rw-r-- 1 5038 1004 2840 Sep  2  2015 /srv/org/wikimedia/releases/debian/dists/trusty-mediawiki/InRelease

https://releases.wikimedia.org/debian/dists/jessie-mediawiki/InRelease got me a blank page it is working now.

I have hit it with curl at least once and in my browser had but after a force refresh I got served the proper content.

I am thus blaming our frontend cache which is tracked in T134989