Would it be possible to install some modern browser with headless capability (Chrome, maybe Chromium or Firefox if licensing issues) on the grid?
Description
Event Timeline
Headless browsers come with a lot of dependencies. I believe we've rejected suggestions for that in the past.
I don't see any dedicated headless packages for chromium/firefox in debian repos or apt.wm.o, CI uses regular chromium/firefox-esr packages, which would involve pulling down a fairly large pile of dependencies.
The use case is https://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/ProcBot_8. No API endpoints exist for editing abusefilters. So I decided to mimic the HTTP requests, but that approach can turn up a few caveats and is slightly less maintainable. I cooked up a web scraping approach using Selenium as well and that might work better. But to run it'll need a browser with headless and modern JS support (both Chrome and Firefox work, so I mentioned those two; either will work).
I once downloaded the Firefox binary and it mostly worked (this was using slimerjs back when that was still usable) - https://www.mozilla.org/en-US/firefox/all/#product-desktop-release
I mean, you can always download the binary to your tool's home dir and see if that will work headlessly without the libraries for x11. That would be much lighter than installing X libraries on everything.
We have Xvfb on the grid too which may or may not be useful in running some browser. The ws-export tool used to use it to run Calibre on the grid.
I'm closing all grid engine install/update tasks as the [[ grid engine is being removed | https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation ]]. Please see the instructions page on how to migrate your tool to Kubernetes. Thanks.