Page MenuHomePhabricator

Support full colour 3D models on Wikimedia projects
Open, Needs TriagePublic

Assigned To
Authored By
John_Cummings
Mar 4 2020, 3:14 PM
Referenced Files
F32412296: image.png
Oct 24 2020, 8:42 AM
Tokens
"Love" token, awarded by Kristbaum."Like" token, awarded by Prototyperspective."Barnstar" token, awarded by Zblace."100" token, awarded by Sj."Love" token, awarded by YavBav09."Like" token, awarded by Strainu."Like" token, awarded by Kozuch."Love" token, awarded by Huntster."Love" token, awarded by AxelPettersson_WMSE."Love" token, awarded by Jony."Like" token, awarded by William_Avery."Like" token, awarded by Ninjastrikers."Mountain of Wealth" token, awarded by NavinoEvans."Party Time" token, awarded by Veracious."Stroopwafel" token, awarded by VIGNERON."Yellow Medal" token, awarded by Esh77."Love" token, awarded by Richard_Nevell."Yellow Medal" token, awarded by ManavpreetKaur."Orange Medal" token, awarded by Pigsonthewing."Stroopwafel" token, awarded by Abbe98."Mountain of Wealth" token, awarded by John_Cummings.

Description

Full colour 3D models on Wikimedia projects would be a really great addition

Content repositories
There are 10,000 of openly licensed full colour 3D models available online including

File format options
There are several open file formats to chose from:

  1. glTF 2.0, some work has already been done on supporting this file format but seems to have stalled, see T187844
  2. AMF supports material, texture, constellation and metadata, allowing you to show full colour models. https://en.wikipedia.org/wiki/Additive_manufacturing_file_format
  3. If supporting AMF isn't possible for a some reason 3MF may be an alternative open file format that could fulfil the same need https://en.wikipedia.org/wiki/3D_Manufacturing_Format
  4. .obj can only display shape information, adding .MTL support would allow .obj to display colour but would be really fiddly and not user friendly at all and would require supporting 2 new file formats

Whichever format is chosen it would be very helpful to allow users to download the model in multiple different formats including .stl which is the most popular format for 3D printing (stl only holds shape information and is already supported by Commons).

Process and skills needed
Steps for adding new file types to Wikimedia Commons are listed here.
Javascript and PHP will be needed to implement this

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Maybe we can acquire one or more voluntary people?

Maybe it is a stupid question, but I want to figure out what we can achieve, because it would be a fantastic addition to Commons.

If you have the ability, feel free to volunteer :) [seriously, if anyone is interested and has some background knowledge in programming and wants to do this, we will provide advice and help]

If you want to recruit someone, probably better to look elsewhere, as everyone on this task is already aware of the request.

I am afraid, I can't, but I will keep my eyes open :)

Would be really nice if we can achieve this

The WMF agreed to do this in principle [so long as it did not use too much in the way of developer time] at the last Wikimania. Perhaps I should reach out to the WMF and remind them of this commitment?

I wrote a short report on my chat with the WMF at the Commons village pump which can be read here; https://commons.wikimedia.org/wiki/Commons:Village_pump/Proposals/Archive/2023/09#Enable_textured_3D_files_on_Commons

Please do!

I also asked some people if they know someone who could contribute.

In T246901#9226554, @brion wrote:

So if we add server-side rendering, we need a clean way to sandbox it, that works both in standalone installs and via Thumbor, and can be relatively sensibly managed and packaged, and made disableable with a switch-flip.

@Legoktm on SJ's computer here :) We can use Shellbox for sandboxing, which transparently handles standalone installs, but someone will need to do the Thumbor integration. I already ported all the deployed MediaHandlers to use Shellbox so there's plenty of examples from others to copy from!

In T246901#9324384, @Sj wrote:
In T246901#9226554, @brion wrote:

So if we add server-side rendering, we need a clean way to sandbox it, that works both in standalone installs and via Thumbor, and can be relatively sensibly managed and packaged, and made disableable with a switch-flip.

@Legoktm on SJ's computer here :) We can use Shellbox for sandboxing, which transparently handles standalone installs, but someone will need to do the Thumbor integration. I already ported all the deployed MediaHandlers to use Shellbox so there's plenty of examples from others to copy from!

Hey, do you have an idea of how to progress this? It would be such a good addition and this is getting quite a vintage phab ticket. Is there a way to add it to someone's list of work at WMF to investigate further and plan the next steps?

Thanks

In T246901#9394722, @Sj wrote:

@brion what do you say?

Probably the wrong account: @bvibber

Still waiting on WMF to budget time to look into doing more multimedia work. :)

Please note there is as far as I know no active Wikimedia-sponsored work at present on any multimedia file format support except for self-driven 20% time by me plus volunteer work by @TheDJ on TimedMediaHandler (audio/video playback).

Problem analysis has not changed since my initial reply years ago:

  • use GLTF format for color models, it's standard and well supported
  • update the frontend and the backend renderers
  • if necessary, replace them with newer libraries

One additional thing: Safari has native USD model support, depending on whether there are format patent issues or not (needs investigation) it might be worth including support for native display via a conversion. This would be a stretch goal, potentially assignable via Mobile Apps if interested for iOS/iPad/Vision Pro compatibility.

I *would* like do a spike test bringing the gltf branch up to date or replacing it with another renderer... I think it's relatively self-contained compared to other multimedia projects and I may be able to squeeze it into my misc tech debt time budget. ;)

Lemme take this for now, within a couple weeks I should either have a better rendrer that's worth pushing forward or have decided it needs more effort spent. :)

(I see this is also in the Hackathon workboard -- perfect timing! I'll have updates on the patch for folks to test by then but I'll be remote for it. :D)

Still waiting on WMF to budget time to look into doing more multimedia work. :)

Please note there is as far as I know no active Wikimedia-sponsored work at present on any multimedia file format support except for self-driven 20% time by me plus volunteer work by @TheDJ on TimedMediaHandler (audio/video playback).

I asked a staff member weeks ago and she said fiscal year ends at the end of June. Maybe then the budget can be used for the implementation, among others? :)

(I see this is also in the Hackathon workboard -- perfect timing! I'll have updates on the patch for folks to test by then but I'll be remote for it. :D)

Is there any news after the Hackathon on this topic? :)

Hi @bvibber thanks so much for picking this up after 4 years, I'm not a wizard so I don't really understand your updates, but if there's anything muggles can do to help with this please let us know :) E.g I'd be happy to start a list of open license repositories for full colour 3d models

Hi @bvibber thanks so much for picking this up after 4 years, I'm not a wizard so I don't really understand your updates, but if there's anything muggles can do to help with this please let us know :) E.g I'd be happy to start a list of open license repositories for full colour 3d models

There you go:

:)

Hi @bvibber thanks so much for picking this up after 4 years, I'm not a wizard so I don't really understand your updates, but if there's anything muggles can do to help with this please let us know :) E.g I'd be happy to start a list of open license repositories for full colour 3d models

There you go:

:)

I can verify that Smithsonian 3D has been waiting a long time for Commons to support files that can do colors/textures

We at the Basque Wikimedians User Group are currently working on a GLAM project which may have content from museums currently available at SketchFab uploaded into Commons. It is known that this is currently impossible.

https://www.blender.org/download/demo-files/ also offers free *.blend files, where colored meshes can be exported

There's many further sources of 3D models with textures such as this. Also note that several STL files should probably get uploaded as a new version for the textured version (example). However, don't know why listing sources would help close the issue.

I found some sort of GLTF Viewer from the Poly Haven source code repository: https://github.com/Poly-Haven/polyhaven.com/tree/master/components/AssetPage/WebGL

Maybe this would help to implement relevant functions?

I have created a prototype for a viewer that also supports GLB files in the ‘mediawiki’ style. Only simple GLB files and KTX2 compression are supported. I am not yet satisfied with the positioning of the models, especially in view of the thumbnails. Help and feedback are very welcome.

You could choose from different free STL and GLB models, or upload your own data. Is KTX2 compression required?

Github Page: https://opendem.github.io/WikiMediaExtension3D_Test_GLB_Format/
Github: https://github.com/OpenDEM/WikiMediaExtension3D_Test_GLB_Format

It can take a while for the larger models to load, no ProgressBar is implemented here.

@OpenDEM

I just took a look at it, and this is already very good! Thank you very much for this :) (Color and shading make a huge difference)

As a feedback I suggest maybe adding key bindings, so this would make it easier to navigate (for beginners), and maybe stats like how many polygons or edges this model has

I uploaded a duck test files, works also fine. Maybe the zooming could be a bit smoother :)

Nice work, @OpenDEM! 😊

I think it’s important to understand why this is taking so long to improve. Here is some context on how we got here and some of the reasons:

  • The Wikimedia Foundation has not allocated enough resources to improve this. As @bvibber stated: "Still waiting on WMF to budget time to look into doing more multimedia work. :)"
  • This topic has not gained enough traction to be included in recent editions of the Community Wishlist. In November 2025 it will be 10 years since it was requested
  • Upgrading these two repositories, 3d2png and 3D extension, is complex for non WMF technical staff. I'm not saying it can’t be done by volunteers, but it requires a certain degree of expertise in WMF's infrastructure.

I have also tried what seems to me the shortest path to get this done: updating the 3d2png Thumbor plugin and the 3D extension (with relative success), and I created this prototype of how a page with a 3D viewer could look. I would be happy to discuss and work together on this. You can join here to discuss it further.

Meanwhile, the 3D content market is consolidating. Epic Games has just launched fab.com (Sketchfab, ArtStation, Unreal Engine Marketplace, Quixel).
A public domain 3D content repository would be of immense value, and WMF should lead in this.

Having a list of 3D repositories is helpful, but I suggest we have a serious discussion about securing more WMF resources for this.

  • Could someone submit a proposal to the Community Wishlist?
  • What other ways might there be to secure more resources for development?

Hope this helps.

Thanks a lot @GabrielLucas for your enlightening background information and all the work! I will approach our local Wikipedia chapter next week and ask for support for the Wishlist.

Very important step :)
Let's make it happen

The current issues surrounding Sketchfab show how important a non-commercial platform for sharing and visualising textured 3D models would be. Click here for the petition with lots of background information: https://www.change.org/p/keep-sketchfab-alive-preserve-open-access-to-3d-art-museum-collections

We added a page on Commons to collect these kinds of examples and the general issue for a more non-technical audience, to not crowd this Phabricator task with usage examples. Please feel free to improve it ;)
https://commons.wikimedia.org/wiki/Commons:Textured_3D

The current issues surrounding Sketchfab show how important a non-commercial platform for sharing and visualising textured 3D models would be. Click here for the petition with lots of background information: https://www.change.org/p/keep-sketchfab-alive-preserve-open-access-to-3d-art-museum-collections

Finally! :)

The Wikimedia Foundation will be hosting a series of community calls to help prioritize support efforts from Wikimedia Foundation for the 2025-2026 Fiscal Year.

The purpose of these calls is to support community members in hearing more from one another - across uploaders, moderators, GLAM enthusiasts, tool and bot makers, etc. - about the future of Commons. There is so much to discuss about the general direction of the project, and we hope that people from different perspectives can think through some of the tradeoffs that will shape Commons going forward.

Our first call will focus on Content Organization. It will take place at two different time slots:

  • The first one will be on November 21, at 08:00 UTC, and it will be hosted on Zoom by Senior Director of Product Management Runa Bhattacharjee; you can subscribe to it on Meta;
  • The second one will be on November 21, at 16:00 UTC, and it will be hosted on Zoom by Chief Product & Technology Officer Selena Deckelmann; you can subscribe to it on Meta.

If you cannot attend the meeting, you are invited to express your point of view at any time you want on the Commons community calls talk page. We will also post the notes of the meeting on the project page, to give the possibility to read what was discussed also to those who couldn’t attend it.

If you want, you are invited to share this invitation with all the people you think might be interested in this call.

just a warm note to @OpenDEM @PantheraLeo1359531 and @GabrielLucas : thanks for the progress and keeping this thread live

Huge thanks from me, too :). Happy to see the project growing

As of today, it is possible to vote on ‘Technische_Wünsche_2024’ for a thematic focus (26.11.2024-9.12.2024). Please vote for 'COMMONS-ERSTELLEN' to bring our cause forward!
https://de.wikipedia.org/wiki/Wikipedia:Umfragen/Technische_W%C3%BCnsche_2024

I've added a few small changes to @OpenDEM's prototype viewer. The default centering/positioning of the 3D models should be more consistent now, but let us know if you find issues!

Would it be helpful to have a document somewhere, writing up requirements and/or best practices on the following considerations ...

  • model orientation (which direction is up, forward)
  • file size
  • performance
  • materials, textures, lighting, and appearance

... for artists and contributors trying to upload 3D models? And if so... what would be the best place to share that?

The focus area including this task / proposal is now open for voting along with a few other Commons-related areas: https://meta.wikimedia.org/wiki/Community_Wishlist/Focus_areas/Media_formats,_editing,_and_display

Hi @donmccurdy, @Prototyperspective,

I'm new to this community, previously Cultural Heritage Lead at sketchfab.com, currently a co-chair of the IIIF 3D Community Group, and very happy to learn about the conversation underway here about updating 3D support on Wikipedia/Wiki Commons.

I'm interested to know how best I might contribute insights from my experience to this conversation?

It feels like there is a lot of overlap with discussions we have had about 3D in the IIIF community, and I have a lot of more practical/anecdotal thoughts that I think/hope would be of value here. I've responded to the topics in the list Don shared below.

I've added a few small changes to @OpenDEM's prototype viewer. The default centering/positioning of the 3D models should be more consistent now, but let us know if you find issues!

Would it be helpful to have a document somewhere, writing up requirements and/or best practices on the following considerations ...

  • model orientation (which direction is up, forward)

For the in-development IIIF 3D spec, the group settled on positive y axis points upwards

  • file size

This is generally subjective based on the subject and model type, but providing recommendations to uploaders is good idea e.g. "5mb total file size is great, 200mb is starting to exclude some audiences". It is also worth thinking about supporting a 'display model' and associated higher resolution 'downloadable versions'.

  • performance

Can you elaborate on what you mean by this? Is there a min. standard for media performance on Wiki platforms?

  • materials, textures, lighting, and appearance

Lots of considerations here, even for 'bare minimum' set up or recommendations. Over in the IIIF group we started with a set of user stories to help guide development, with the acknowledgement that the userbase of Wikipedia/Commons is much broader than 'cultural heritage' users.

... for artists and contributors trying to upload 3D models? And if so... what would be the best place to share that?

Hi @Nebulousflynn, nice to have you :). Afaik it is about the implementation of a 3D viewer for textured models and what "quality of life" features it should include, also while handling larger files.

I have added the current status of the ongoing work to the page https://commons.wikimedia.org/wiki/Commons:Textured_3D. Help is very welcome.

Would it be helpful to have a document somewhere, writing up requirements and/or best practices on the following considerations ...

model orientation (which direction is up, forward)
file size
performance
materials, textures, lighting, and appearance
... for artists and contributors trying to upload 3D models? And if so... what would be the best place to share that?

For other formats we usually try to downscale if the original asset is too large. However That might not be possible/practical here.

The best place to discuss and document these things would be at commons. These can develop separately over time of course as the community gains experience.

... for artists and contributors trying to upload 3D models? And if so... what would be the best place to share that?

I think Commons:3D models would be a good place. That page is really sparse now, but we could be inspired by Commons:Video and Commons:Audio.

Hi everyone,

Today we have taken another step forward to improve the support of 3D models with more advanced features (color, animations, etc) in Wikipedia.

@OpendDEM and I have been testing the latest updates to the thumbnail generator 3d2png and the 3D extension and it can be said that it has been a success and we have been able to see the first GLB files in operation.

If someone is an expert in threejs or the GLTF standard and wants to help in the development, they can get in touch in the Telegram group,

You can see an animated gif (image quality and lighting is better in the 3D viewer) here: https://ibb.co/P69vfTm

Hi @Nebulousflynn, good to see you here! Broadly agreed on your comments. One special concern on file size and performance — it's quite possible to create "small" 3D files (<5MB) with performance or memory problems bad enough to freeze a webpage. I think some guidelines or even hard restrictions in the uploader might be helpful in that regard. The Khronos Group maintains a set of guidelines for publishing "real-time" 3D assets (that is, models intended to be rendered on a consumer device, interactively, as opposed to offline in a GPU farm somewhere) at Realtime Asset Creation Guidelines that I'd consider a very good starting point.

The glTF file format is specialized for real-time 3D publishing use cases, but even so... user-provided models are likely to require some optimization (either manually or as part of the upload process). Perhaps we can work out details in the Commons: 3D models page as @Ainali suggests.

Hi, as we have some options on how to proceed with WikiMedia 3D we would
like to invite you to a video conference. You can find the proposal for
the Agenda here: Commons:Textured_3D.
Feel free to edit and add to it. Date would be Sunday 23.02.2025 at 16:00 UTC.
Get in contact for the dial-in link for Jitsi (contact@opendemdata.info) or join the Telegram Group.

Date would be Sunday 16.02.2025 at 16:00 UTC.

Didn't you mix up the Sundays? What we discussed in the Telegram chat was Sunday the 23rd, not the 16th.

Sorry, you're absolutely right, it's 23 February 2025 and we'll be meeting via Jitsi.

I wish you success in this project to support 3D images with texture in Wikimedia (glb file).
I have a lot of 3D images of heritage monuments on Sketchfab (over 2500) and I'd be very interested in putting them in Wikimedia, which is better suited to this style of image than Fab.com.
I don't have the skills to help with development, but perhaps I can be of help with user interface testing once development is well advanced.

Hi everyone,
I follow this thread since the beginning and I wanted to contribute more.
I think that bringing 3D into Commons is very important but also need a bit of attention.
I have a (very) long experience in 3D scanning for cultural heritage and I have contributed to the field.

A disclaimer: 3D objects have multiple uses inside wikimedia, explaining concepts, illustrating complex structures, geometry explaiations etc.
I feel confident in bringing hopefully useful suggestions only for what concern the use of 3D objects for illustrating cultural heritage artifacts.
I know the field well and I have contributed to writing official guidelines at country level.
So be indulgent if I can have a CH biased point of view :)

@ALoopingIcon

Nice to have you here :)
Textured meshes will be a very important part in the future represeting real-life objects, evolved from simple photographs (and from there to panoramics). And it makes it easier to preserve precious objects :D

Yesterday I had a meeting with the responsible for heritage at the Basque Government. She told me that they are trying to have lots of "digital twins" (i.e. 3D objects), but they don't have a good place to have them online. We neither! Just imagine the possibilities.

Hi @ALoopingIcon, if you would like to contribute something to the development system, please write to me for the access data: contact@opendemdata.info

A disclaimer: 3D objects have multiple uses inside wikimedia, explaining concepts, illustrating complex structures, geometry explaiations etc.
I feel confident in bringing hopefully useful suggestions only for what concern the use of 3D objects for illustrating cultural heritage artifacts.

Sorry, you're absolutely right, it's 23 February 2025 and we'll be meeting via Jitsi.

how does one get an invite and link to the meeting..thanks

Here is the link to the video conference:  https://konferenz.buehl.digital/mediawiki3d
The conference system is based on Jitsi.
 Sunday 23.02.2025 at 16:00 UTC

it would be very good to have someone from WMF joining. @bvibber

so once on telegram..what's the channel to talk to this group? or link or whatever magic to use telegram? thanks