In order to replace the ingress-nginx controller which is going EOL, we need to fox tools currently relying on custom Ingress object annotations to implement custom functionality.
Description
Description
Details
Details
Related Changes in Gerrit:
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| P:toolforge: k8s: haproxy: Handle plain toolforge.org domain | operations/puppet | production | +2 -2 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T392356 Replace ingress-nginx before upstream EOL date | |||
| Resolved | taavi | T415192 Move ingress annotations to an allowlist model | |||
| Open | None | T414674 Remove remaining uses of ingress-nginx specific annotations | |||
| In Progress | Feature | bd808 | T414836 Create a reusable container to replace nginx ingress anonymizing reverse proxy setups |
Event Timeline
Comment Actions
The current list of tools relying on this functionality is at P87548. Of those, a couple are simple redirects that can be replaced with https://wikitech.wikimedia.org/wiki/Tool:Containers#Redirect_container, but the ones proxying to external upstream servers will need an another solution.
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2026-01-15T12:49:21Z] <taavi> migrate to tool-containers/redirect T414674
Comment Actions
Mentioned in SAL (#wikimedia-cloud) [2026-01-15T12:50:45Z] <taavi> migrate to tool-containers/redirect T414674
Comment Actions
Change #1227321 had a related patch set uploaded (by Majavah; author: Majavah):
[operations/puppet@production] P:toolforge: k8s: haproxy: Handle plain toolforge.org domain
Comment Actions
Change #1227321 merged by Majavah:
[operations/puppet@production] P:toolforge: k8s: haproxy: Handle plain toolforge.org domain
Comment Actions
Triaging as High as the EOL date is soon:
In March 2026, Ingress NGINX maintenance will be halted, and the project will be retired. After that time, there will be no further releases, no bugfixes, and no updates to resolve any security vulnerabilities that may be discovered. The GitHub repositories will be made read-only and left available for reference.
Comment Actions
I will try to make a reverse proxy container in the same spirit as the redirect container. I can borrow from things I learned building https://gitlab.wikimedia.org/toolforge-repos/gitlab-content to make something relatively light weight I think.
Comment Actions
That would be very helpful, thank you!
Do you have thoughs already on how that would deal with tools that currently host a "normal" web service in addition to those reverse proxies? (e.g. tool-moedata has four proxy ingresses in addition to the mail webservice, it seems.)
Comment Actions
I was assuming that any Kubernetes Ingress solution should allow routing a path to a service. The jobs service can handle starting a continuous job and exposing a port controlled by that job as a Service object. I think we can then still specify an Ingress that routes a subset of a tool's traffic to that Service based on the request path. There are some Ingress objects like the proxy-scdn Ingress in the bd808-test tool that are using pathType: ImplementationSpecific in their routing, but I think this has generally been done to simplify configuration by specifying a regex like /scdn(/|$)(.*) for the handled path rather than adding two Ingress objects where one handles the /scdn exact path and another handles the /scdn/ prefix path. If we find an instance that has some more complicated need than the exact + prefix combo the easiest answer might end up being embedding a reverse proxy within the main webservice, but that's just a guess until I see a complicated use case in practice.