Page MenuHomePhabricator

Evaluate shared Chromium service (WebSocket) for 3d2png and chromium-render
Open, Needs TriagePublic

Description

3d2png needs Chromium as a new dependency for GLB rendering. Rather than bundling it independently into each service, the proposal is to run Chromium as a standalone container exposed via WebSocket, and have both 3d2png and chromium-render connect to it. This would also require migrating chromium-render to use the WebSocket interface, and updating 3d2png to use puppeteer + chromium in the same way.

For this to work without increasing maintenance burden, we'd need CI checks validating both services still work after each Chromium bump, since version updates can introduce breaking changes.

The multipart container image approach was also considered, but that would still require rebuilding both services after base image changes and isn't automated, so the separate container with external socket access seems like the cleaner path forward.

Open Questions (based on my research)

  • In the current setup, you get a new Chromium instance per request. With Chromium as a service, each request gets a new page. Not sure how Chromium handles pages that time out or error out. Would it kill the instance?
  • How resilient would this be under heavy load? For example, batch uploading 3D models to Commons? Is any load balancing required?
  • What would CI look like? At the very least we should trigger dependent service tests, and we have to make sure the tests are comprehensive enough to be trusted.
  • Do we have any metrics on how many requests we get to chromium-render across our projects?

Any help answering this questions, and/or defining a path forward is greatly appreciated.

Event Timeline

A while back, I contacted https://www.glb2png.com (commercial GLB thumbnailer) to ask about their system architecture:

"My solution (GLB 2 PNG) also uses multiple servers in the background that run Puppeteer. However, I have optimized the software running on the servers to generate images as reliably, efficiently, and—above all—quickly as possible. To achieve this, I use load balancers, download queues, caching of the GLB models, and a CDN to avoid rendering the same image again if it has already been requested." (Translated)

Fortunately, we have been promised support, so it would make sense to strengthen our ties.

I think there could be a lot of traffic, especially right after the new format is released. We’ve backed up terabytes of free GLB models from Sketchfab that are waiting to be imported. However, preview images are already available there. I think an API that supports the automatic upload of all metadata and data would be useful here.