Based on the below guide, I think the best/most standard way to store the API credentials for the application, would be inside a file in the project's data directory.
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Managing_and_sharing_files

The /data/scratch directory would be ideal, if I had sophisticated enough deployment orchestration to create the credentials file upon every deployment. Barring that though, persisting the data in the project folder seems like the next-best option.

On second thought, the scratch directory isn't a good idea. According to the below page, it's shared across all instances in all projects.

https://wikitech.wikimedia.org/wiki/Help:Shared_storage#/data/scratch

On third thought:
Running this web app inside of kubernetes complicates things, because my image won't have access to the shared storage locations out of the box.

According to here: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Building_container_images#Using_NFS_shared_storage
"We recommend using the envvars service to create environment variables with secret configuration values like passwords instead of reading configuration files from NFS."

Based on this, sounds like I'll have a much easier time if I simply use environment variables rather than trying to force in a config file. I'll investigate that next - seems like the way to go.

The production OAuth consumer is setup here: https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/1ef316396fb537bfdd76a0c1051d2e98
The code for this is in this commit: https://gitlab.wikimedia.org/toolforge-repos/wd-act/-/commit/572ed9d065cba64ebfde0cc885bc795d86df421c

We've got a playbook now to configure the required environment variables in the toolforge hosting environment.

Only thing left is to update the maintainer documentation with info about how to stitch all this together to configure the runtime environment. Also need to figure out how to properly store/share the configuration details with maintainers long-term.