Page MenuHomePhabricator

Have 3d2png work with RoomEnvironment on threejs v162 or migrate to puppeteer if possible.
Open, Stalled, LowPublic

Description

I've tried implementing RoomEnvironment without success. Even with https://github.com/donmccurdy/3D2PNG_GLB_Support/blob/97f379af74c95649e63426f60bbaaa38d6d1b03d/PMREMGenerator.js it seems WEBGL_color_buffer_float is not supported by headless-gl(?) I'm unclear as to why this doesn't work. It fails silently.

https://github.com/OpenDEM/3D2PNG_GLB_Support does not work for me locally and I get the same result. Objects are rendered fully black.

Possible path forward: (if no solution is found)
Migrate rendering from headless-gl to Puppeteer because PMREMGenerator (used by RoomEnvironment) requires float/half-float framebuffer support, which is not available in headless-gl (afaikt) but is supported in chromium. This will also allow us to use threejs latest.
It's worth noting that 3d2png runs on thumbor's app image, this means we need to add chromium as a dependency if not already there. Not sure what the implications of this can be or if at all allowed or possible given our current infrastructure.

Migrating to puppeteer seems to work on my rudimentary testing.

Event Timeline

@hnowlan who would be the right person to talk to about moving away from headless-gl and migrate to puppeteer considering that 3d2png runs on thumbor's image? Not sure if this is at all possible.

No, this ist just working for some very Limited cases. WE should move away From headless-gl!

Hi @dmaza – is this a new feature request for the 3D extension? Please provide a bit more context and we can triage appropriately.

HSwan-WMF moved this task from Incoming/Inbox to Backlog on the Reader Growth Team board.

Hi @dmaza – is this a new feature request for the 3D extension? Please provide a bit more context and we can triage appropriately.

Sorry I missed this, This is related to the work Commtech is doing to support GLTF on commons. I figured the 3D tag is appropriate but I did not mean to get your team involved.

Dear @dmaza @egardner @OpenDEM,

As you have figured out headless-gl route is a hits a dead end as it doesn´t support WEBGL2.

I have been working on it and created to projects exploring other two possible routes for a 3D headless render:

C++ / FIlament - https://github.com/gabriel-lucas/3d2png-C
Rust / WGPU - https://github.com/gabriel-lucas/3d2png-rust

They are not finished yet, they need more development.
The idea is to support all the GLTF standard extensions.
I have been testing them against the GLTF assets library - https://github.khronos.org/glTF-Assets/

I share them, so we could analyze possible routes for improving 3D image rendering.
There are some aspects to consider like: proper fit to mediawiki infrastructure, library dependencies, performance, security and so on.

Glad to discuss all with you

As far as I can tell, puppetter.js is already part of the MediaWiki infrastructure: https://www.mediawiki.org/wiki/Proton
I see the advantages in its independence from data formats and viewers. We’ve already listed the pros and cons here: https://commons.wikimedia.org/wiki/Commons:Textured_3D

@GabrielLucas We are having internal discussions on how we can piggyback on Proton (see above) and move towards a Puppeteer + Chromium approach, but what you are working on looks very neat. You can track our progress at T424872: Evaluate shared Chromium service (WebSocket) for 3d2png and chromium-render. In the meantime, we will have a fallback thumbnail for GLB files.

Removing inactive assignee