Page MenuHomePhabricator

Make eventstreams-internal available to WMF staff without an ssh tunnel
Open, Needs TriagePublic

Description

eventstreams-internal is a non-public deployment of EventStreams that has access to all Event Platform streams. Currently can only be accessed by users with a WMF production ssh account via ssh tunneling.

Exposing this at a public domain with proper auth would allow Data Platform users to explore stream documentation and schemas using OpenAPI docs (e.g. this), as well as view live stream data in their browsers.


Original ticket description from Luca:

While deploying the new version of eventstreams, I noticed that the internal endpoint seems not used in ages:

https://grafana.wikimedia.org/d/znIuUcsWz/eventstreams?orgId=1&refresh=1m&var-dc=codfw%20prometheus%2Fk8s&var-service=eventstreams-internal&from=now-6M&to=now

From the logs on logstash (both eqiad and codfw) I don't see anything relevant either, maybe I am missing something but I am wondering if we should undeploy to simplify the maintenance.

Let me know!

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Good question. I had expected Product teams to use this more often, but perhaps the ssh tunnel barrier is enough for them to never do so.

Or, perhaps its existence is too hidden in docs and they don't know about it! Let's ask Experimentation Lab and Product-Analytics what they think.

Ottomata added subscribers: BTullis, mpopov, phuedx.

@BTullis do you think it would be possible to add authentication and a public domain to this service? I think Metrics Platform folks would really like this. cc @mpopov @phuedx (This would allow you to use a stream.wikimedia.org UI but for all internal event streams).

Ottomata moved this task from Backlog to Stream Data Products on the Event-Platform board.
Ottomata moved this task from Stream Data Products to Components on the Event-Platform board.
Ottomata added a subscriber: Ahoelzl.
Ottomata renamed this task from Traffic for eventstreams-internal seems to be zero for the past months to Make eventstreams-internal available to WMF staff without an ssh tunnel.Oct 25 2024, 1:43 PM
Ottomata updated the task description. (Show Details)

@BTullis do you think it would be possible to add authentication and a public domain to this service?

Yes, I think that would be quite feasible. We could use the CAS/SSO implementation and authenticate to it using the OIDC protocol, as we are with Superset and Airflow and DataHub and MPIC.
There would need to be an LDAP group to whom the rights would be given, equivalent to the analytics-privatedata-users POSIX group, I suppose. We are already configuring a number of new LDAP groups with cross-checking in T375729, so I can't see a big problem with this part.

Then the next question would be where to do the authentcation.
We could either:

...or

  • add an authenticating reverse proxy using envoy or some other kind of service.

At first glance, the envoy based solution looks pretty neat and tidy, given that we already have envoy installed in every pod.

  • add an authenticating reverse proxy using envoy or some other kind of service.

At first glance, the envoy based solution looks pretty neat and tidy, given that we already have envoy installed in every pod.

Envoy Gateway is a different thing from Envoy Proxy -- the former is a management layer around the latter. We only run Envoy Proxy.

But we're using oauth2-proxy on k8s in the aux cluster to front trace.wikimedia.org and it's been working fine for that -- link to our config. Other teams also use it on bare metal for access to Thanos and a few other pieces of infra.

But we're using oauth2-proxy on k8s in the aux cluster to front trace.wikimedia.org and it's been working fine for that -- link to our config. Other teams also use it on bare metal for access to Thanos and a few other pieces of infra.

Nice! Thanks for that info.

Would it be valuable to move MPIC to using oauth2-proxy for consistency with these other systems?