Page MenuHomePhabricator

mw:thumbor swift user doesn't have access to wikipedia-commons-local-temp.* swift containers
Closed, ResolvedPublic

Description

I'm not sure what happened, but on thumbor1001 I can't even stat any of those containers, I get a 403 for all of them.

Spot-checking a couple, temp containers of other wikis don't have that problem.

Details

Related Gerrit Patches:
mediawiki/core : masterRemove unnecessary slash in scalerThumbUrl
mediawiki/core : wmf/1.33.0-wmf.24Remove unnecessary slash in scalerThumbUrl
mediawiki/core : wmf/1.33.0-wmf.24Add support for X-Swift-Secret to upload stash
mediawiki/core : masterAdd support for X-Swift-Secret to upload stash
operations/debs/python-thumbor-wikimedia : masterUpgrade to 2.4
operations/software/thumbor-plugins : masterVersion bump
operations/software/thumbor-plugins : masterPass Swift secret in test_temp
operations/debs/python-thumbor-wikimedia : masterUpgrade to 2.3
operations/software/thumbor-plugins : masterTreat temp containers as private

Event Timeline

Gilles triaged this task as High priority.Apr 6 2019, 12:40 PM
Gilles created this task.
Gilles added a project: SRE-swift-storage.

Me and @jijiki took a look at this today, here's the findings so far:

  • The user that got access is mw:thumbor-private, not mw:thumbor e.g.:
root@ms-fe1005:~# swift stat wikipedia-commons-local-temp.f7 
         Account: AUTH_mw
       Container: wikipedia-commons-local-temp.f7
         Objects: 174
           Bytes: 1760233378
        Read ACL: mw:thumbor-private,mw:media
       Write ACL: mw:thumbor-private,mw:media
         Sync To:
        Sync Key:
   Accept-Ranges: bytes
      X-Trans-Id: txa378b73ca75e4db08290d-005cab17ae
X-Storage-Policy: standard
   Last-Modified: Thu, 03 Jan 2019 19:51:04 GMT
     X-Timestamp: 1381944283.41528
    Content-Type: text/plain; charset=utf-8
  • Same (wrong) permissions in codfw/eqiad

With the above, my hunch is that ACLs for commons temp containers got mw:thumbor-private and not mw:thumbor by way of maint scripts in mediawiki, the reason though is unclear to me.

setZoneAccess's intent seems to be to make the temp container private, just list a private wiki's containers: https://github.com/wikimedia/mediawiki-extensions-WikimediaMaintenance/commit/72259a740611331797a59ebf4675d1eadc6215aa

That's perfectly fine. Uploads in the temp containers are indeed meant to only be accessed by the uploader, with MediaWiki acting as the gatekeeper via its authentication.

I think the only thing we need to fix here is:

  • tweak the logic in thumbor-plugins' image.py so that "temp" and "deleted" induce self.context.private
  • run setZoneAccess on every wiki, which will change the temp container permissions to mw:thumbor-private

Basically when we made thumbor compatible with private wikis, we overlooked the fact that on regular wikis "temp" (and "deleted") are treated the same way in terms of privacy and auth by setZoneAccess.

Gilles claimed this task.Apr 8 2019, 12:23 PM
Gilles added a project: Performance-Team.
Gilles added a subscriber: jijiki.

Also, it's possible that MediaWiki will need some follow-up changes for the UploadWizard case specifically, to ensure that it sends the secret header in that situation, as it's probably a different codepath than when dealing with a private wiki. We'll find out once we've taken care of Thumbor and the ACL udpates.

Change 502206 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Treat temp containers as private

https://gerrit.wikimedia.org/r/502206

Change 502206 merged by Gilles:
[operations/software/thumbor-plugins@master] Treat temp containers as private

https://gerrit.wikimedia.org/r/502206

Change 488060 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.3

https://gerrit.wikimedia.org/r/488060

Change 488060 merged by Gilles:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.3

https://gerrit.wikimedia.org/r/488060

Change 502520 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@master] Add support for X-Swift-Secret to upload stash

https://gerrit.wikimedia.org/r/502520

Change 502521 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Pass Swift secret in test_temp

https://gerrit.wikimedia.org/r/502521

Hi, @Gilles . Any chance that T219679 might be related to this problem?

Gilles added a comment.Apr 9 2019, 3:50 PM

Yes, that's why it's the parent task.

Change 502521 merged by Gilles:
[operations/software/thumbor-plugins@master] Pass Swift secret in test_temp

https://gerrit.wikimedia.org/r/502521

Change 502523 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/software/thumbor-plugins@master] Version bump

https://gerrit.wikimedia.org/r/502523

Change 502523 merged by Gilles:
[operations/software/thumbor-plugins@master] Version bump

https://gerrit.wikimedia.org/r/502523

Change 502526 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.4

https://gerrit.wikimedia.org/r/502526

Change 502526 merged by Gilles:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 2.4

https://gerrit.wikimedia.org/r/502526

Yes, that's why it's the parent task.

Ah! I just noticed that. Thanks! :)

Change 502520 merged by jenkins-bot:
[mediawiki/core@master] Add support for X-Swift-Secret to upload stash

https://gerrit.wikimedia.org/r/502520

Change 502532 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@wmf/1.33.0-wmf.24] Add support for X-Swift-Secret to upload stash

https://gerrit.wikimedia.org/r/502532

Change 502532 merged by jenkins-bot:
[mediawiki/core@wmf/1.33.0-wmf.24] Add support for X-Swift-Secret to upload stash

https://gerrit.wikimedia.org/r/502532

Change 502541 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@master] Remove unnecessary slash in scalerThumbUrl

https://gerrit.wikimedia.org/r/502541

Change 502542 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@wmf/1.33.0-wmf.24] Remove unnecessary slash in scalerThumbUrl

https://gerrit.wikimedia.org/r/502542

Change 502542 merged by Gilles:
[mediawiki/core@wmf/1.33.0-wmf.24] Remove unnecessary slash in scalerThumbUrl

https://gerrit.wikimedia.org/r/502542

Mentioned in SAL (#wikimedia-operations) [2019-04-09T17:04:16Z] <gilles@deploy1001> Synchronized php-1.33.0-wmf.24/includes/specials/SpecialUploadStash.php: T220265 Add support for X-Swift-Secret to upload stash (duration: 00m 53s)

Change 502541 merged by jenkins-bot:
[mediawiki/core@master] Remove unnecessary slash in scalerThumbUrl

https://gerrit.wikimedia.org/r/502541

Mentioned in SAL (#wikimedia-operations) [2019-04-10T08:12:20Z] <gilles> T220265 foreachwiki extensions/WikimediaMaintenance/filebackend/setZoneAccess.php --backend local-multiwrite

Mentioned in SAL (#wikimedia-operations) [2019-04-10T10:46:12Z] <gilles> T220265 setZoneAccess on all wikis finished

Gilles closed this task as Resolved.Apr 10 2019, 10:46 AM