Page MenuHomePhabricator

Ensure thumbor container access is preserved by mw filebackend setzoneaccess
Closed, ResolvedPublic

Description

when adding a wiki (https://wikitech.wikimedia.org/wiki/Add_a_wiki) the maintanenace script to set swift containers ACL is run, we should make sure rw access for "mw:thumbor" is kept if present (and added if not?) for thumbnail containers

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 1 2016, 9:58 AM

Where does the code for that live?

Gilles lowered the priority of this task from Normal to Low.Nov 15 2016, 12:30 PM

@Gilles it is in WikimediaMaintenance

Also related, the temp containers should have their access updated (crf T150760)

fgiunchedi raised the priority of this task from Low to High.Aug 6 2017, 5:46 PM

Since thumbor is in production now I'm bumping the priority because container perms need to be correct for new wikis

fgiunchedi renamed this task from ensure thumbor container access is preserved by mw filebackend setzoneaccess to Ensure thumbor container access is preserved by mw filebackend setzoneaccess.Aug 6 2017, 5:46 PM

Ping? Not granting thumbor access for newly created wikis it means files uploaded there won't get thumbnails.

Gilles claimed this task.Aug 18 2017, 4:17 PM
Gilles added a comment.EditedSep 5 2017, 7:48 AM

Looking at the puppet code, I see that thumbor uses ::swift::user and ::swift::key. I presume that these have different values on thumbor machines than mediawiki ones thanks to some private hiera data.

In other words, running under mwscript on Terbium, addWiki.php shouldn't have access to those values, should it? Because it has its own Mediawiki username and key populated in those hiera keys. I.e. it has no way to know what swift username is used on the thumbor hosts and can't open the rights on behalf of thumbor.

Therefore it seems to me like adding a step to https://wikitech.wikimedia.org/wiki/Add_a_wiki detailing how to add rw access for the thumbor user is unavoidable. And if we're going to do that, why not just have that happen after the addWiki.php step?

I see 2 alternatives:

  • Thumbor hosts using the same Swift username and key as Mediawiki (I understand why you've decided not to do that)
  • Store the Thumbor Swift credentials under different hiera key names and expose the username to Mediawiki, which is then applied on container creation in SwiftFileBackend.php. This will make container access definition for the Thumbor user piggy-back on addWiki.php. This requires exposing the Thumbor Swift username in $wmfSwiftEqiadConfig and $wmfSwiftCodfwConfig which I believe resides in an Ops repo I don't have access to, where the private keys of things for the PHP config are defined.

I assume that last alternative is what you really want.

So first step would be for you to do that, expose the Thumbor Swift username in a new key (let's say 'swiftThumborUser') on the wmfSwiftEqiadConfig and wmfSwiftCodfwConfig arrays and then I can work on modifying Mediawiki core so it can consume 'swiftThumborUser'.

Gilles reassigned this task from Gilles to fgiunchedi.Sep 5 2017, 7:48 AM
Gilles claimed this task.Sep 5 2017, 3:49 PM

Change 376043 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/mediawiki-config@master] Expose Thumbor swift username

https://gerrit.wikimedia.org/r/376043

Change 376251 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@master] Give thumbor swift user/w access to containers

https://gerrit.wikimedia.org/r/376251

Change 376251 merged by jenkins-bot:
[mediawiki/core@master] Give thumbor swift user r/w access to containers

https://gerrit.wikimedia.org/r/376251

Change 376043 merged by jenkins-bot:
[operations/mediawiki-config@master] Expose Thumbor swift username

https://gerrit.wikimedia.org/r/376043

Mentioned in SAL (#wikimedia-operations) [2017-09-26T13:25:32Z] <hashar@tin> Synchronized wmf-config/filebackend.php: Expose Thumbor swift username - T144479 (duration: 00m 44s)

Gilles closed this task as Resolved.Sep 26 2017, 1:26 PM

The change will only be effective once the train has happened, but I think this is all we need for the next time a wiki is created.

This requires exposing the Thumbor Swift username in $wmfSwiftEqiadConfig and $wmfSwiftCodfwConfig which I believe resides in an Ops repo I don't have access to, where the private keys of things for the PHP config are defined.

That'd come from /srv/mediawiki-staging/private/PrivateSettings.php which all deployers should have access to

This requires exposing the Thumbor Swift username in $wmfSwiftEqiadConfig and $wmfSwiftCodfwConfig which I believe resides in an Ops repo I don't have access to, where the private keys of things for the PHP config are defined.

That'd come from /srv/mediawiki-staging/private/PrivateSettings.php which all deployers should have access to

I did set it there and deploy it, just forgot to mention it here