when adding a wiki (https://wikitech.wikimedia.org/wiki/Add_a_wiki) the maintanenace script to set swift containers ACL is run, we should make sure rw access for "mw:thumbor" is kept if present (and added if not?) for thumbnail containers
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Expose Thumbor swift username | operations/mediawiki-config | master | +12 -4 | |
| Give thumbor swift user r/w access to containers | mediawiki/core | master | +32 -14 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T165585 Make creating a new Language project easier | |||
| Open | None | T158730 Automate WMF wiki creation | |||
| Resolved | • Gilles | T121388 Service-based thumbnailing re-architecture in production with Thumbor | |||
| Resolved | • Gilles | T144479 Ensure thumbor container access is preserved by mw filebackend setzoneaccess |
Event Timeline
Comment Actions
Since thumbor is in production now I'm bumping the priority because container perms need to be correct for new wikis
Comment Actions
Ping? Not granting thumbor access for newly created wikis it means files uploaded there won't get thumbnails.
Comment Actions
Looking at the puppet code, I see that thumbor uses ::swift::user and ::swift::key. I presume that these have different values on thumbor machines than mediawiki ones thanks to some private hiera data.
In other words, running under mwscript on Terbium, addWiki.php shouldn't have access to those values, should it? Because it has its own Mediawiki username and key populated in those hiera keys. I.e. it has no way to know what swift username is used on the thumbor hosts and can't open the rights on behalf of thumbor.
Therefore it seems to me like adding a step to https://wikitech.wikimedia.org/wiki/Add_a_wiki detailing how to add rw access for the thumbor user is unavoidable. And if we're going to do that, why not just have that happen after the addWiki.php step?
I see 2 alternatives:
- Thumbor hosts using the same Swift username and key as Mediawiki (I understand why you've decided not to do that)
- Store the Thumbor Swift credentials under different hiera key names and expose the username to Mediawiki, which is then applied on container creation in SwiftFileBackend.php. This will make container access definition for the Thumbor user piggy-back on addWiki.php. This requires exposing the Thumbor Swift username in $wmfSwiftEqiadConfig and $wmfSwiftCodfwConfig which I believe resides in an Ops repo I don't have access to, where the private keys of things for the PHP config are defined.
I assume that last alternative is what you really want.
So first step would be for you to do that, expose the Thumbor Swift username in a new key (let's say 'swiftThumborUser') on the wmfSwiftEqiadConfig and wmfSwiftCodfwConfig arrays and then I can work on modifying Mediawiki core so it can consume 'swiftThumborUser'.
Comment Actions
Change 376043 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/mediawiki-config@master] Expose Thumbor swift username
Comment Actions
Change 376251 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/core@master] Give thumbor swift user/w access to containers
Comment Actions
Change 376251 merged by jenkins-bot:
[mediawiki/core@master] Give thumbor swift user r/w access to containers
Comment Actions
Change 376043 merged by jenkins-bot:
[operations/mediawiki-config@master] Expose Thumbor swift username
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-26T13:25:32Z] <hashar@tin> Synchronized wmf-config/filebackend.php: Expose Thumbor swift username - T144479 (duration: 00m 44s)
Comment Actions
The change will only be effective once the train has happened, but I think this is all we need for the next time a wiki is created.
Comment Actions
That'd come from /srv/mediawiki-staging/private/PrivateSettings.php which all deployers should have access to