Page MenuHomePhabricator

S3 credentials for the WDQS K8S deployment
Closed, ResolvedPublic

Description

We received S3 credentials in T425973 and have successfully used them locally to create and interact with buckets, both manually and via an Airflow DAG (T426764).

Now we would like to use these credentials to access an S3 bucket from a Kubernetes container in DSE.

A draft for the corresponding helm chart exists as a CR. In there we currently reference the credentials in values.yaml:

init:
  image: wdqs-backend-init
  version: latest
  config:
    public:
      KAFKA_GROUP: wdqs-updater-${WDQS_NODE_NAME}
      ...
    private:
      S3_ACCESS_KEY: accesskey
      S3_SECRET_KEY: privatekey

The two keys on the bottom are the ones in question. As of this writing the chart is in draft and does surely contain issues. Feedback on how to use the secret better would be welcome.
For a quick overview of the targeted architecture please see the WDQS Gitlab overview page and/or simply ping the Wikidata platform team.

Event Timeline

trueg updated the task description. (Show Details)
bking changed the task status from Open to In Progress.May 28 2026, 7:40 PM
bking claimed this task.
bking triaged this task as Medium priority.

I'll take a closer look tomorrow, but from what I can tell, the secret is being used correctly. I'm comparing the Airflow helm chart's handling of secrets against the secret creation in your linked CR , and they look identical.

I will be out for 10 days after tomorrow, but my general advice would be to merge the CR (and the patches that create the namespaces) as soon as y'all are in agreement with the implementation, then iterate rapidly using a homedir deploy .

bking changed the task status from In Progress to Open.May 28 2026, 10:05 PM
BTullis renamed this task from S3 credentials for the WDQS K82 deployment to S3 credentials for the WDQS K8S deployment.Jun 2 2026, 3:07 PM

@brouberol any pending work on your end? Can we resolve this task?

Sorry, I was out of PTO. I was wondering the same. AFAICT the work was done, and just not reflected here. Closing.