| Ottomata | |
| Mar 7 2023, 4:59 PM |
| F37621770: image.png | |
| Aug 23 2023, 11:10 AM |
@nettrom_WMF asked if there was any way for him to view running presto query and/or cluster usage. The Presto UI is a nice way to do this.
Can we configure presto and ferm to allow access to the Presto UI port from stat boxes, and then update instructions here to tunnel via stat boxes?
I think that there would be a privacy concern if we were to do this.
There's no authentication or authorization in the Presto UI (See https://github.com/prestodb/presto/issues/11298) so if we were to enable access then all users of the stat boxes would be able to view (and kill) all of each others queries.
At the moment, only members of the analytics-admins groups have this right, by virtue of being able to SSH into the coordinator and accessing the Presto UI on localhost.
@JAllemandou - @Milimetric - @mforns - What do you think? Would you agree that this behaviour of seeing and managing other users' queries might be problematic?
I note that Trino have got around this problem by adding an authentication and access control mechanism to their UI: https://trino.io/docs/current/admin/web-interface.html#authentication
I'm pretty sure that presto doesn't have these features yet: https://prestodb.io/docs/current/admin/web-interface.html
We can come back to this request if we complete T266640: Decide whether to migrate from Presto to Trino and decide to migrate from Presto to Trino. Until then I suspect that we don't have any other choice but to decline this task.
Given the proposed solution from Andrew I don't think there'd be more privacy issues than with Hadoop, i.e: cluster users can see others folks jobs. If the presto UI is hidden behind an ssh tunnel I'm fine with it.
Thanks @JAllemandou. Just to confirm, users would be able to kill each others' jobs as well from the Presto UI.
We have similar issues with yarn, although a bit different since we differenciate prod users, but for regular users, IIRC one can kill another job.
I'm uncomfortable adding more dependencies on SSH tunnels, and exposing a full admin interface just to get visibility on workloads. I'm declining this for now. If there is a strong case to extend the feature we offer, let's make that case and see what solution we can come up with.