For commons/mediawiki given the huge number of files we shard across 256 containers based on the filename hash e.g. container.ff. This helps both with contention on containers and "blast radius" e.g. if something happens to a single container not all files are affected.
Implementing something like this requires aid from the application of course, what's your expectation in terms of number of files in this case?

What happens with large bodies here? e.g. clients ask for "big" files ?

Indeed, this is something I will look into. Thanks for the reminder!

things go boom when the 2gb limit is exceeded. I'm currently investigating the underlyting APIs, there is at least theoretical support for streaming the data instead of processing it all at once, however, it's of course undocumented and somewhat complicated by the layers of tech involved.

Ok actually it would appear that I spoke too soon regarding large files. The chunking of the files happens at a higher level in the code - so the back end storage driver doesn't need to worry about large files - they are always sharded into a bunch of smaller objects before being passed to the storage engine. It appears that chunking is handled in the PhabricatorFileUploadSource class.

Ok now I shared across a bunch of containers based on the first two bytes of entropy which phabricator is already generating in order to give files a random names.

It looks like getContainer is never invoked with an empty key (and shouldn't, afaics?)

How would container creation and setting access work? The way MW handles it is creating the required containers on wiki setup and setting permissions for public/private containers with setZoneAccess.php. Is all file access mediated/proxied by Phabricator in this case, thus all containers can be private to the Phabricator account?

indeed this is true

As of now it attempts to create the container on demand and does not override the default access control. Indeed all file access is proxied through phabricator so the containers can be private.

I think it'd make more sense to have $key non-optional instead and container sharding either enabled or disabled, not based on key length

Thanks! Ok I think auto-creation will work for now. A useful addition/feature in the future would be to not require auto-creation, i.e. through an utility that would create the required containers and can be run with a different set of swift credentials. This way the swift account Phabricator uses doesn't need to be privileged in any way, just rw access to selected containers.

Could we just create the containers with a shell script on the swift cluster? ;) I could create a utility to do it but building that inside phabricator's UI is a lot of boilerplate for not much benefit.

In my experience shell scripts are never "just", so no, better avoiding that :) In this case what I had in mind with "utility" isn't a full fledged UI, a php command line utility that reads phabricator config is what I had in mind.

Also when (if? don't know if you have plans to upstream) this gets accepted upstream, said utility is useful to other users too and contained in Phabricator already. Anyways, something to think about, auto-creation is fine for now and only slightly more complex for users that want to create containers beforehand.