Me and @jijiki took a look at this today, here's the findings so far:

• The user that got access is mw:thumbor-private, not mw:thumbor e.g.:

root@ms-fe1005:~# swift stat wikipedia-commons-local-temp.f7 
         Account: AUTH_mw
       Container: wikipedia-commons-local-temp.f7
         Objects: 174
           Bytes: 1760233378
        Read ACL: mw:thumbor-private,mw:media
       Write ACL: mw:thumbor-private,mw:media
         Sync To:
        Sync Key:
   Accept-Ranges: bytes
      X-Trans-Id: txa378b73ca75e4db08290d-005cab17ae
X-Storage-Policy: standard
   Last-Modified: Thu, 03 Jan 2019 19:51:04 GMT
     X-Timestamp: 1381944283.41528
    Content-Type: text/plain; charset=utf-8

• Same (wrong) permissions in codfw/eqiad

• The Last-Modified timestamp is around the time testcommonswiki might have been created in T197616: Create a production test wiki in group0 to parallel Wikimedia Commons (and/or setZoneAccess.php run) and spot-checking the commons temp containers Last-Modified, the timing is close enough to one another that looks like a script ran through the containers and changed them.

With the above, my hunch is that ACLs for commons temp containers got mw:thumbor-private and not mw:thumbor by way of maint scripts in mediawiki, the reason though is unclear to me.

setZoneAccess's intent seems to be to make the temp container private, just list a private wiki's containers: https://github.com/wikimedia/mediawiki-extensions-WikimediaMaintenance/commit/72259a740611331797a59ebf4675d1eadc6215aa

That's perfectly fine. Uploads in the temp containers are indeed meant to only be accessed by the uploader, with MediaWiki acting as the gatekeeper via its authentication.

I think the only thing we need to fix here is:

• tweak the logic in thumbor-plugins' image.py so that "temp" and "deleted" induce self.context.private
• run setZoneAccess on every wiki, which will change the temp container permissions to mw:thumbor-private

Basically when we made thumbor compatible with private wikis, we overlooked the fact that on regular wikis "temp" (and "deleted") are treated the same way in terms of privacy and auth by setZoneAccess.

Also, it's possible that MediaWiki will need some follow-up changes for the UploadWizard case specifically, to ensure that it sends the secret header in that situation, as it's probably a different codepath than when dealing with a private wiki. We'll find out once we've taken care of Thumbor and the ACL udpates.

Change 502206 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Treat temp containers as private

https://gerrit.wikimedia.org/r/502206

Change 502206 merged by Gilles:
[operations/software/thumbor-plugins@master] Treat temp containers as private

https://gerrit.wikimedia.org/r/502206

Change 488060 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.3

https://gerrit.wikimedia.org/r/488060

Change 488060 merged by Gilles:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.3

https://gerrit.wikimedia.org/r/488060

Change 502520 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@master] Add support for X-Swift-Secret to upload stash

https://gerrit.wikimedia.org/r/502520

Change 502521 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Pass Swift secret in test_temp

https://gerrit.wikimedia.org/r/502521

Hi, @Gilles . Any chance that T219679 might be related to this problem?

Yes, that's why it's the parent task.

Change 502521 merged by Gilles:
[operations/software/thumbor-plugins@master] Pass Swift secret in test_temp

https://gerrit.wikimedia.org/r/502521

Change 502523 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Version bump

https://gerrit.wikimedia.org/r/502523

Change 502523 merged by Gilles:
[operations/software/thumbor-plugins@master] Version bump

https://gerrit.wikimedia.org/r/502523

Change 502526 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.4

https://gerrit.wikimedia.org/r/502526

Change 502526 merged by Gilles:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.4

https://gerrit.wikimedia.org/r/502526

In T220265#5097886, @Gilles wrote:

Yes, that's why it's the parent task.

Ah! I just noticed that. Thanks! :)

Change 502520 merged by jenkins-bot:
[mediawiki/core@master] Add support for X-Swift-Secret to upload stash

https://gerrit.wikimedia.org/r/502520

Change 502532 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@wmf/1.33.0-wmf.24] Add support for X-Swift-Secret to upload stash

https://gerrit.wikimedia.org/r/502532

Change 502532 merged by jenkins-bot:
[mediawiki/core@wmf/1.33.0-wmf.24] Add support for X-Swift-Secret to upload stash

https://gerrit.wikimedia.org/r/502532

Change 502541 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@master] Remove unnecessary slash in scalerThumbUrl

https://gerrit.wikimedia.org/r/502541

Change 502542 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@wmf/1.33.0-wmf.24] Remove unnecessary slash in scalerThumbUrl

https://gerrit.wikimedia.org/r/502542

Change 502542 merged by Gilles:
[mediawiki/core@wmf/1.33.0-wmf.24] Remove unnecessary slash in scalerThumbUrl

https://gerrit.wikimedia.org/r/502542

Mentioned in SAL (#wikimedia-operations) [2019-04-09T17:04:16Z] <gilles@deploy1001> Synchronized php-1.33.0-wmf.24/includes/specials/SpecialUploadStash.php: T220265 Add support for X-Swift-Secret to upload stash (duration: 00m 53s)

Change 502541 merged by jenkins-bot:
[mediawiki/core@master] Remove unnecessary slash in scalerThumbUrl

https://gerrit.wikimedia.org/r/502541

Mentioned in SAL (#wikimedia-operations) [2019-04-10T08:12:20Z] <gilles> T220265 foreachwiki extensions/WikimediaMaintenance/filebackend/setZoneAccess.php --backend local-multiwrite

Mentioned in SAL (#wikimedia-operations) [2019-04-10T10:46:12Z] <gilles> T220265 setZoneAccess on all wikis finished