Page MenuHomePhabricator

Support full colour 3D models on Wikimedia projects
Open, Needs TriagePublic

Tokens
"Love" token, awarded by Huntster."Love" token, awarded by AxelPettersson_WMSE."Love" token, awarded by Jony."Like" token, awarded by William_Avery."Like" token, awarded by Ninjastrikers."Mountain of Wealth" token, awarded by NavinoEvans."Party Time" token, awarded by Veracious."Stroopwafel" token, awarded by VIGNERON."Yellow Medal" token, awarded by Esh77."Love" token, awarded by Richard_Nevell."Yellow Medal" token, awarded by ManavpreetKaur."Orange Medal" token, awarded by Pigsonthewing."Stroopwafel" token, awarded by Abbe98."Mountain of Wealth" token, awarded by Mrjohncummings.
Assigned To
None
Authored By
Mrjohncummings
Mar 4 2020, 3:14 PM

Description

Full colour 3D models on Wikimedia projects would be a really great addition

Content repositories
There are 10,000 of openly licensed full colour 3D models available online including

File format options
There are several open file formats to chose from:

  1. glTF 2.0, some work has already been done on supporting this file format but seems to have stalled, see T187844
  2. AMF supports material, texture, constellation and metadata, allowing you to show full colour models. https://en.wikipedia.org/wiki/Additive_manufacturing_file_format
  3. If supporting AMF isn't possible for a some reason 3MF may be an alternative open file format that could fulfil the same need https://en.wikipedia.org/wiki/3D_Manufacturing_Format
  4. .obj can only display shape information, adding .MTL support would allow .obj to display colour but would be really fiddly and not user friendly at all and would require supporting 2 new file formats

Whichever format is chosen it would be very helpful to allow users to download the model in multiple different formats including .stl which is the most popular format for 3D printing (stl only holds shape information and is already supported by Commons).

Process and skills needed
Steps for adding new file types to Wikimedia Commons are listed here.
Javascript and PHP will be needed to implement this

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Yes please, models with textures would be fantastic

Esh77 awarded a token.Mar 12 2020, 8:50 AM
VIGNERON added a subscriber: VIGNERON.
Mrjohncummings renamed this task from Support AMF 3d file format on Commons to allow full colour 3d models on Wikimedia projects to Support full colour 3D models on Wikimedia projects .Mar 12 2020, 9:09 AM
Mrjohncummings updated the task description. (Show Details)
Mrjohncummings updated the task description. (Show Details)
Aklapper renamed this task from Support full colour 3D models on Wikimedia projects to Support full colour 3D models on Wikimedia projects in AMF or 3MF file format.Mar 12 2020, 10:24 AM

I can't find it right now, but I recall having a list of requirements for new file formats to be support. Top of my head:

  1. Has to be a free format (no patents or other problems)
  2. We need to be able to render it somehow
  3. We need to be able to do a sanity check on the file (to prevent malicious content)

Answering these questions will probably increase the chance of the file format actually being supported on Commons

Spinster added a subscriber: Spinster.EditedMar 12 2020, 11:04 AM

Multichill has good points above - I think I recall that there are some solid practical reasons why our 3D feature does not support textures yet.

That said, I have heard frequently from GLAMs that they don't find the current non-textured 3D support interesting at all - they want to provide the textures. So definitely +1'ing the demand.

David_Haskiya_WMSE added a comment.EditedMar 12 2020, 12:22 PM

+1 for the demand (but think glTF may be the more suitable format (but am not religious about it))

Let me illustrate:
Swedish museums have increasingly started producing 3D-models. Artefacts mostly, but also buildings, monuments and archaeological sites. Many Swedish GLAMs do this under an open license and many also already share 2D-images on Commons. Where do they share their 3D models? Sketchfab and, in a few cases, Scan the World - MyMiniFactory. Not Commons. Why? Because we can't display textures they feel we don't do the models or the real life objects and environments they represent justice.

There have been previous discussions on the subject of sanitation of 3D formats over at T131723.

The version of glTF that support bundled textures is essentially a binary (with the gitf.bin file extension), so the technical barrier for implementation is likely higher I would think.

Mrjohncummings renamed this task from Support full colour 3D models on Wikimedia projects in AMF or 3MF file format to Support full colour 3D models on Wikimedia projects.Mar 12 2020, 12:56 PM
Mrjohncummings updated the task description. (Show Details)

I can't find it right now, but I recall having a list of requirements for new file formats to be support. Top of my head:

  1. Has to be a free format (no patents or other problems)
  2. We need to be able to render it somehow
  3. We need to be able to do a sanity check on the file (to prevent malicious content)

Answering these questions will probably increase the chance of the file format actually being supported on Commons

Thanks,

  1. All the file formats listed in the task description are open

2 and 3, I can't answer these but I think that 3 is which gITF was stalled?

  1. We need to be able to do a sanity check on the file (to prevent malicious content)

I think that 3 is which gITF was stalled?

Yes. In a quick conversation, item 3 was deemed effectively impossible to do automatically in a way that was "good enough" that it wouldn't increase the review burden on Commons sysops (even more so than videos do).

That doesn't mean we'd never do this, but it'd need active buy-in from the community that they were aware of the possibilities for abuse and OK with the additional work.

Mrjohncummings added a comment.EditedMar 12 2020, 4:14 PM
  1. We need to be able to do a sanity check on the file (to prevent malicious content)

I think that 3 is which gITF was stalled?

Yes. In a quick conversation, item 3 was deemed effectively impossible to do automatically in a way that was "good enough" that it wouldn't increase the review burden on Commons sysops (even more so than videos do).

That doesn't mean we'd never do this, but it'd need active buy-in from the community that they were aware of the possibilities for abuse and OK with the additional work.

@Jdforrester-WMF Thanks very much for the information, what about it made it difficult assess? I'm wondering if one of the other formats doesn't have this issue and could be implemented more easily.

Also what additional work would it need in practice?

  1. We need to be able to do a sanity check on the file (to prevent malicious content)

I think that 3 is which gITF was stalled?

Yes. In a quick conversation, item 3 was deemed effectively impossible to do automatically in a way that was "good enough" that it wouldn't increase the review burden on Commons sysops (even more so than videos do).

That doesn't mean we'd never do this, but it'd need active buy-in from the community that they were aware of the possibilities for abuse and OK with the additional work.

@Jdforrester-WMF Thanks very much for the information, what about it made it difficult assess? I'm wondering if one of the other formats doesn't have this issue and could be implemented more easily.

Also what additional work would it need in practice?

Sorry, I was unclear. This isn't a problem with any given rich 3D format, it's a problem with the nature of things. For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

Right now, a Commons sysop can glance at new image uploads as they come in and determine in a fraction of a second if it's problematic (e.g. copyvios or CSIA or other issues); for videos, they have to watch at least some of it; for 3D models, they'd need to interact with it to examine it from lots of different viewpoints/etc. It's just a different scale in terms of work.

  1. We need to be able to do a sanity check on the file (to prevent malicious content)

I think that 3 is which gITF was stalled?

Yes. In a quick conversation, item 3 was deemed effectively impossible to do automatically in a way that was "good enough" that it wouldn't increase the review burden on Commons sysops (even more so than videos do).

That doesn't mean we'd never do this, but it'd need active buy-in from the community that they were aware of the possibilities for abuse and OK with the additional work.

@Jdforrester-WMF Thanks very much for the information, what about it made it difficult assess? I'm wondering if one of the other formats doesn't have this issue and could be implemented more easily.

Also what additional work would it need in practice?

Sorry, I was unclear. This isn't a problem with any given rich 3D format, it's a problem with the nature of things. For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

Right now, a Commons sysop can glance at new image uploads as they come in and determine in a fraction of a second if it's problematic (e.g. copyvios or CSIA or other issues); for videos, they have to watch at least some of it; for 3D models, they'd need to interact with it to examine it from lots of different viewpoints/etc. It's just a different scale in terms of work.

@Jdforrester-WMF ok, thanks very much, to make sure I understand the issue I can summarise and you tell me if this is correct? It isn't a problem with any specific file format itself, its the fact that colour 3d models are complex and uploaders could 'hide' other images etc inside the model. The blocker is that there is no automatic way to look for these hidden things and there needs to be some kind of agreed process on Commons for rules around uploading these file. Once these rules are agreed then WMF or someone else could then work on the actual plugin or whatever is needed to allow upload and viewing of the files on Commons, Wikipedia etc. Is this correct?

@Jdforrester-WMF Great, thanks, I'm happy to work on a community consensus for supporting full colour 3D models, do you have any suggestion of the process to use for this and any best practice examples I could follow?

@Jdforrester-WMF Great, thanks, I'm happy to work on a community consensus for supporting full colour 3D models, do you have any suggestion of the process to use for this and any best practice examples I could follow?

No. To my knowledge, nothing like this has ever been attempted.

@Jdforrester-WMF best of luck to us.... should it be done through an RFC?

@Jdforrester-WMF best of luck to us.... should it be done through an RFC?

I'm not sure. It's not the Foundation's call how the Commons community wants to come to decisions, after all.

@Jdforrester-WMF best of luck to us.... should it be done through an RFC?

I'm not sure. It's not the Foundation's call how the Commons community wants to come to decisions, after all.

@Jdforrester-WMF of course, I was asking because you have a lot of experience with community and technical development, I can ask around and see who has suggestions. Thanks for all your help with this, I'll let you know how I get on with things

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

I think definitely this should be brought up as a possible solution for 3. in a community discussion. It essentially removes the burden from the sysops by only assigning the rights to users in good standing.

I also recall another more technical but yet similar reason, and that is that some 3D-formats allows for scripts also (which could possibly have malicious purposes). But I guess that is easier in a way by just saying that we will not support them at all (yet).

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

I think definitely this should be brought up as a possible solution for 3. in a community discussion. It essentially removes the burden from the sysops by only assigning the rights to users in good standing.

I also recall another more technical but yet similar reason, and that is that some 3D-formats allows for scripts also (which could possibly have malicious purposes). But I guess that is easier in a way by just saying that we will not support them at all (yet).

Does anyone know how or where the MP3 discussions happened?

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

I think definitely this should be brought up as a possible solution for 3. in a community discussion. It essentially removes the burden from the sysops by only assigning the rights to users in good standing.

I also recall another more technical but yet similar reason, and that is that some 3D-formats allows for scripts also (which could possibly have malicious purposes). But I guess that is easier in a way by just saying that we will not support them at all (yet).

Does anyone know how or where the MP3 discussions happened?

A few discussions are linked from the following page: https://commons.wikimedia.org/wiki/User:CKoerner_(WMF)/MP3_patrol_discussion

Sorry, I was unclear. This isn't a problem with any given rich 3D format, it's a problem with the nature of things. For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

Right now, a Commons sysop can glance at new image uploads as they come in and determine in a fraction of a second if it's problematic (e.g. copyvios or CSIA or other issues); for videos, they have to watch at least some of it; for 3D models, they'd need to interact with it to examine it from lots of different viewpoints/etc. It's just a different scale in terms of work.

That was actually not the kind of sanity check I was thinking about when I wrote above list. I was more thinking about a technical sanity check for things like embedded executables or a virus. What you are describing is the nature of textures of a 3D object. If we as the Commons community allow textures, this seems to be a reasonable implication. The way to verify these files would be to have an easy tool to unpack all the textures and show them as tiles in the browser. Than you can just inspect a bunch of tiles.
No blockers for technical sanity checking here?

Sorry, I was unclear. This isn't a problem with any given rich 3D format, it's a problem with the nature of things. For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

Right now, a Commons sysop can glance at new image uploads as they come in and determine in a fraction of a second if it's problematic (e.g. copyvios or CSIA or other issues); for videos, they have to watch at least some of it; for 3D models, they'd need to interact with it to examine it from lots of different viewpoints/etc. It's just a different scale in terms of work.

That was actually not the kind of sanity check I was thinking about when I wrote above list. I was more thinking about a technical sanity check for things like embedded executables or a virus. What you are describing is the nature of textures of a 3D object. If we as the Commons community allow textures, this seems to be a reasonable implication. The way to verify these files would be to have an easy tool to unpack all the textures and show them as tiles in the browser. Than you can just inspect a bunch of tiles.
No blockers for technical sanity checking here?

@Multichill thanks for the explanation, a couple of questions:

  • Is there an example of a previous phabricator task or similar for this kind of assessment for another format or extension or something? I'm hoping and assuming there is an established process we could follow for dong this part
  • Who could do this assessment? Both in terms of who has the technical knowledge and also who can give an 'ok' for something like this

Maybe the simplest thing would be for you to write a phabricator task which is a sub task of this one? I'm very hapy to put work into making this happen but the technical side is not something I can do much on except being a hype man

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

I think definitely this should be brought up as a possible solution for 3. in a community discussion. It essentially removes the burden from the sysops by only assigning the rights to users in good standing.

I also recall another more technical but yet similar reason, and that is that some 3D-formats allows for scripts also (which could possibly have malicious purposes). But I guess that is easier in a way by just saying that we will not support them at all (yet).

Does anyone know how or where the MP3 discussions happened?

A few discussions are linked from the following page: https://commons.wikimedia.org/wiki/User:CKoerner_(WMF)/MP3_patrol_discussion

Thanks very much @Abbe98, I can't see from here where a decision was made about either supporting MP3 or the decision to restrict access to uploading MP3s, maybe admin noticeboard or something?

@Jdforrester-WMF best of luck to us.... should it be done through an RFC?

I'm not sure. It's not the Foundation's call how the Commons community wants to come to decisions, after all.

I would reccomend starting a thread on COM:VPP

I can't find it right now, but I recall having a list of requirements for new file formats to be support. Top of my head:

  1. Has to be a free format (no patents or other problems)
  2. We need to be able to render it somehow
  3. We need to be able to do a sanity check on the file (to prevent malicious content)

Answering these questions will probably increase the chance of the file format actually being supported on Commons

I would add to that: it should be self-contained. Textures or whatever should be in the file - it should not link to textures or subresourses in other files or elsewhere on the internet.

I would add to that: it should be self-contained. Textures or whatever should be in the file - it should not link to textures or subresourses in other files or elsewhere on the internet.

I agree that linking elsewhere on the internet should not happen, but being totally self-contained has no intrinsic value. On the contrary, it makes little sense having to store them separately in each file that are using them as well. This will lead to data duplication in possibly quite a large scale. Therefore, all files already on Commons should be eligible as textures in a 3D-file.

Yupik added a subscriber: Yupik.Mar 14 2020, 1:21 PM
Ayack added a subscriber: Ayack.Mar 14 2020, 5:32 PM

For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

While this may be so, how is it an different to stenographically hidden material in 2D-images or video files?

I agree that linking elsewhere on the internet should not happen, but being totally self-contained has no intrinsic value. On the contrary, it makes little sense having to store them separately in each file that are using them as well. This will lead to data duplication in possibly quite a large scale. Therefore, all files already on Commons should be eligible as textures in a 3D-file.

I'm saying its a technical requirement, because things become much more complicated if we allow cross-file linking. Its unlikely (but not impossible) we would allow cross-file linking in files uploaded to wikimedia sites. (Its ok if the format supports it, we just would need to filter such uploads). The same analysis basically applies to svg.

For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

While this may be so, how is it an different to stenographically hidden material in 2D-images or video files?

Not all that different, in principle, but the nature of 3D makes it an exponentially more difficult problem. It's also why this:

The way to verify these files would be to have an easy tool to unpack all the textures and show them as tiles in the browser. Than you can just inspect a bunch of tiles.

...would be insufficient as a means of verification.

Think of any of the optical illusions you've seen where two images become something completely different when combined at the correct angle. Looking at either of the component images in isolation, or removed from the 3D placement that creates the illusion, nothing would look amiss. But, view the model as it's "intended" (by the malicious designer) to be viewed, and "Oh, there's the genitalia." (To pluck a low-hanging fruit of an example.)

...would be insufficient as a means of verification.

Think of any of the optical illusions you've seen where two images become something completely different when combined at the correct angle. Looking at either of the component images in isolation, or removed from the 3D placement that creates the illusion, nothing would look amiss. But, view the model as it's "intended" (by the malicious designer) to be viewed, and "Oh, there's the genitalia." (To pluck a low-hanging fruit of an example.)

While that is a risk, I think to a certain extent its one we have to live with, and should just, you know, delete/relabel inappropriate media when we find it. Vandalism both subtle and otherwise has always been a threat, if we deemed it an unacceptable threat we wouldn't be able to have a wiki website at all. [But ultimately, i think this is a discussion best done on wiki ensuring the admins who would have to deal with the situation fully participate]

I understand the risk of having something inappropriate, but I think the risk is really low and whenever a wikimedian or commonist find it it would be deleted or solved. I have been looking to the examples given in the description and it would be something really valuable to have at Wikipedia articles.

To move the discussion on... is there any non-social blocker to do this?

I understand the risk of having something inappropriate, but I think the risk is really low and whenever a wikimedian or commonist find it it would be deleted or solved. I have been looking to the examples given in the description and it would be something really valuable to have at Wikipedia articles.

To move the discussion on... is there any non-social blocker to do this?

@Theklan thanks, I think a couple of things I'd like to do before the discussion

  • Outline exactly the process for implementing this after the community discussion, what work work would need to be done and roughly who would be capable of doing it and create phabcricator tasks for them

And for the discussion

  • Plan the structure of the discussion and what will be explained and what will be asked
  • Collate information on the educational impact of full colour 3D models
  • Collate a fuller list of good quality 3d model sources

Anything else for the discussion you think is needed?

Thanks again

TheDJ added a subscriber: TheDJ.EditedMar 16 2020, 5:02 PM

I can't find it right now, but I recall having a list of requirements for new file formats to be support. Top of my head:

That list would be https://www.mediawiki.org/wiki/Manual:Adding_support_for_new_filetypes

Jony awarded a token.Mar 17 2020, 5:18 AM
Jony added a subscriber: Jony.

@TheDJ thanks very much :)

Mrjohncummings added a subscriber: Keegan.EditedMar 17 2020, 9:24 AM

@Keegan do you know what kinds of programming skills/languages would be needed to implement this?

Knowledge of that file format; 3D extension uses Javascript and PHP.

Mrjohncummings updated the task description. (Show Details)
Mrjohncummings updated the task description. (Show Details)

@Keegan do you know what kinds of programming skills/languages would be needed to implement this?

If you (or anyone) is actually interested in doing the technical work for this, feel free to reach out with any questions. I'm not offering to do it, but I am offering to answer any questions about how code base works, if anyone gets "stuck" trying to do it.

@Bawolff thats very helpful, thanks very much, is there anything you could add to the task description that would help people to get started/not get stuck as easily?

@brion worked on glTF support a while ago. Looks to me like it got quite far, maybe possible to rebase it and take a stab at it.

https://gerrit.wikimedia.org/r/c/3d2png/+/415495

FeRDNYC removed a subscriber: FeRDNYC.Mar 17 2020, 9:30 PM

@brion could you describe what the issues you ran into were and how people could help?

Thanks

Huntster added a subscriber: Huntster.

As a note of support, I'd very much love to see support for glTF. There are so many amazing full colour models of NASA spacecraft, such as https://mars.nasa.gov/resources/24584/curiosity-rover-3d-model/