In T246901#5964613, @Jdforrester-WMF wrote:

In T246901#5964466, @Mrjohncummings wrote:

In T246901#5964446, @Jdforrester-WMF wrote:

In T246901#5963761, @Mrjohncummings wrote:

In T246901#5963328, @Multichill wrote:

3. We need to be able to do a sanity check on the file (to prevent malicious content)

I think that 3 is which gITF was stalled?

Yes. In a quick conversation, item 3 was deemed effectively impossible to do automatically in a way that was "good enough" that it wouldn't increase the review burden on Commons sysops (even more so than videos do).

That doesn't mean we'd never do this, but it'd need active buy-in from the community that they were aware of the possibilities for abuse and OK with the additional work.

@Jdforrester-WMF Thanks very much for the information, what about it made it difficult assess? I'm wondering if one of the other formats doesn't have this issue and could be implemented more easily.

Also what additional work would it need in practice?

Sorry, I was unclear. This isn't a problem with any given rich 3D format, it's a problem with the nature of things. For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

Right now, a Commons sysop can glance at new image uploads as they come in and determine in a fraction of a second if it's problematic (e.g. copyvios or CSIA or other issues); for videos, they have to watch at least some of it; for 3D models, they'd need to interact with it to examine it from lots of different viewpoints/etc. It's just a different scale in terms of work.

@Jdforrester-WMF ok, thanks very much, to make sure I understand the issue I can summarise and you tell me if this is correct? It isn't a problem with any specific file format itself, its the fact that colour 3d models are complex and uploaders could 'hide' other images etc inside the model. The blocker is that there is no automatic way to look for these hidden things and there needs to be some kind of agreed process on Commons for rules around uploading these file. Once these rules are agreed then WMF or someone else could then work on the actual plugin or whatever is needed to allow upload and viewing of the files on Commons, Wikipedia etc. Is this correct?

Yes.

@Jdforrester-WMF Great, thanks, I'm happy to work on a community consensus for supporting full colour 3D models, do you have any suggestion of the process to use for this and any best practice examples I could follow?

In T246901#5964951, @Mrjohncummings wrote:

@Jdforrester-WMF Great, thanks, I'm happy to work on a community consensus for supporting full colour 3D models, do you have any suggestion of the process to use for this and any best practice examples I could follow?

No. To my knowledge, nothing like this has ever been attempted.

@Jdforrester-WMF best of luck to us.... should it be done through an RFC?

In T246901#5964961, @Mrjohncummings wrote:

@Jdforrester-WMF best of luck to us.... should it be done through an RFC?

I'm not sure. It's not the Foundation's call how the Commons community wants to come to decisions, after all.

In T246901#5965005, @Jdforrester-WMF wrote:

In T246901#5964961, @Mrjohncummings wrote:

@Jdforrester-WMF best of luck to us.... should it be done through an RFC?

I'm not sure. It's not the Foundation's call how the Commons community wants to come to decisions, after all.

@Jdforrester-WMF of course, I was asking because you have a lot of experience with community and technical development, I can ask around and see who has suggestions. Thanks for all your help with this, I'll let you know how I get on with things

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

In T246901#5965141, @Abbe98 wrote:

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

I think definitely this should be brought up as a possible solution for 3. in a community discussion. It essentially removes the burden from the sysops by only assigning the rights to users in good standing.

I also recall another more technical but yet similar reason, and that is that some 3D-formats allows for scripts also (which could possibly have malicious purposes). But I guess that is easier in a way by just saying that we will not support them at all (yet).

In T246901#5965272, @Ainali wrote:

In T246901#5965141, @Abbe98 wrote:

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

I think definitely this should be brought up as a possible solution for 3. in a community discussion. It essentially removes the burden from the sysops by only assigning the rights to users in good standing.

I also recall another more technical but yet similar reason, and that is that some 3D-formats allows for scripts also (which could possibly have malicious purposes). But I guess that is easier in a way by just saying that we will not support them at all (yet).

Does anyone know how or where the MP3 discussions happened?

In T246901#5965504, @Mrjohncummings wrote:

In T246901#5965272, @Ainali wrote:

In T246901#5965141, @Abbe98 wrote:

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

I think definitely this should be brought up as a possible solution for 3. in a community discussion. It essentially removes the burden from the sysops by only assigning the rights to users in good standing.

I also recall another more technical but yet similar reason, and that is that some 3D-formats allows for scripts also (which could possibly have malicious purposes). But I guess that is easier in a way by just saying that we will not support them at all (yet).

Does anyone know how or where the MP3 discussions happened?

A few discussions are linked from the following page: https://commons.wikimedia.org/wiki/User:CKoerner_(WMF)/MP3_patrol_discussion

In T246901#5964613, @Jdforrester-WMF wrote:

Sorry, I was unclear. This isn't a problem with any given rich 3D format, it's a problem with the nature of things. For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

Right now, a Commons sysop can glance at new image uploads as they come in and determine in a fraction of a second if it's problematic (e.g. copyvios or CSIA or other issues); for videos, they have to watch at least some of it; for 3D models, they'd need to interact with it to examine it from lots of different viewpoints/etc. It's just a different scale in terms of work.

That was actually not the kind of sanity check I was thinking about when I wrote above list. I was more thinking about a technical sanity check for things like embedded executables or a virus. What you are describing is the nature of textures of a 3D object. If we as the Commons community allow textures, this seems to be a reasonable implication. The way to verify these files would be to have an easy tool to unpack all the textures and show them as tiles in the browser. Than you can just inspect a bunch of tiles.
No blockers for technical sanity checking here?

In T246901#5966708, @Multichill wrote:

In T246901#5964613, @Jdforrester-WMF wrote:

Sorry, I was unclear. This isn't a problem with any given rich 3D format, it's a problem with the nature of things. For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

Right now, a Commons sysop can glance at new image uploads as they come in and determine in a fraction of a second if it's problematic (e.g. copyvios or CSIA or other issues); for videos, they have to watch at least some of it; for 3D models, they'd need to interact with it to examine it from lots of different viewpoints/etc. It's just a different scale in terms of work.

That was actually not the kind of sanity check I was thinking about when I wrote above list. I was more thinking about a technical sanity check for things like embedded executables or a virus. What you are describing is the nature of textures of a 3D object. If we as the Commons community allow textures, this seems to be a reasonable implication. The way to verify these files would be to have an easy tool to unpack all the textures and show them as tiles in the browser. Than you can just inspect a bunch of tiles.
No blockers for technical sanity checking here?

@Multichill thanks for the explanation, a couple of questions:

• Is there an example of a previous phabricator task or similar for this kind of assessment for another format or extension or something? I'm hoping and assuming there is an established process we could follow for dong this part
• Who could do this assessment? Both in terms of who has the technical knowledge and also who can give an 'ok' for something like this

Maybe the simplest thing would be for you to write a phabricator task which is a sub task of this one? I'm very hapy to put work into making this happen but the technical side is not something I can do much on except being a hype man

In T246901#5965714, @Abbe98 wrote:

In T246901#5965504, @Mrjohncummings wrote:

In T246901#5965272, @Ainali wrote:

In T246901#5965141, @Abbe98 wrote:

Could a solution similar to the one for MP3 files make sense? It would allow GLAMs to share vetted 3D content while still lock abusers out.

(To upload a MP3 file to Commons you need a specific user right.)

I think definitely this should be brought up as a possible solution for 3. in a community discussion. It essentially removes the burden from the sysops by only assigning the rights to users in good standing.

I also recall another more technical but yet similar reason, and that is that some 3D-formats allows for scripts also (which could possibly have malicious purposes). But I guess that is easier in a way by just saying that we will not support them at all (yet).

Does anyone know how or where the MP3 discussions happened?

A few discussions are linked from the following page: https://commons.wikimedia.org/wiki/User:CKoerner_(WMF)/MP3_patrol_discussion

Thanks very much @Abbe98, I can't see from here where a decision was made about either supporting MP3 or the decision to restrict access to uploading MP3s, maybe admin noticeboard or something?

In T246901#5965005, @Jdforrester-WMF wrote:

In T246901#5964961, @Mrjohncummings wrote:

@Jdforrester-WMF best of luck to us.... should it be done through an RFC?

I'm not sure. It's not the Foundation's call how the Commons community wants to come to decisions, after all.

I would reccomend starting a thread on COM:VPP

In T246901#5963328, @Multichill wrote:

I can't find it right now, but I recall having a list of requirements for new file formats to be support. Top of my head:

1. Has to be a free format (no patents or other problems)
2. We need to be able to render it somehow
3. We need to be able to do a sanity check on the file (to prevent malicious content)

Answering these questions will probably increase the chance of the file format actually being supported on Commons

I would add to that: it should be self-contained. Textures or whatever should be in the file - it should not link to textures or subresourses in other files or elsewhere on the internet.

In T246901#5969238, @Bawolff wrote:

I would add to that: it should be self-contained. Textures or whatever should be in the file - it should not link to textures or subresourses in other files or elsewhere on the internet.

I agree that linking elsewhere on the internet should not happen, but being totally self-contained has no intrinsic value. On the contrary, it makes little sense having to store them separately in each file that are using them as well. This will lead to data duplication in possibly quite a large scale. Therefore, all files already on Commons should be eligible as textures in a 3D-file.

In T246901#5964613, @Jdforrester-WMF wrote:

For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

While this may be so, how is it an different to stenographically hidden material in 2D-images or video files?

I agree that linking elsewhere on the internet should not happen, but being totally self-contained has no intrinsic value. On the contrary, it makes little sense having to store them separately in each file that are using them as well. This will lead to data duplication in possibly quite a large scale. Therefore, all files already on Commons should be eligible as textures in a 3D-file.

I'm saying its a technical requirement, because things become much more complicated if we allow cross-file linking. Its unlikely (but not impossible) we would allow cross-file linking in files uploaded to wikimedia sites. (Its ok if the format supports it, we just would need to filter such uploads). The same analysis basically applies to svg.

In T246901#5969679, @Pigsonthewing wrote:

In T246901#5964613, @Jdforrester-WMF wrote:

For WP:BEANS reasons I won't go into depth about the potential abuse vectors, but certainly things we touched on were around problematic imagery or content, especially forms that are stenographically "hidden" or textures painted onto the inside of a surface so they're invisible on casual inspection.

While this may be so, how is it an different to stenographically hidden material in 2D-images or video files?

Not all that different, in principle, but the nature of 3D makes it an exponentially more difficult problem. It's also why this:

In T246901#5966708, @Multichill wrote:

The way to verify these files would be to have an easy tool to unpack all the textures and show them as tiles in the browser. Than you can just inspect a bunch of tiles.

...would be insufficient as a means of verification.

Think of any of the optical illusions you've seen where two images become something completely different when combined at the correct angle. Looking at either of the component images in isolation, or removed from the 3D placement that creates the illusion, nothing would look amiss. But, view the model as it's "intended" (by the malicious designer) to be viewed, and "Oh, there's the genitalia." (To pluck a low-hanging fruit of an example.)

...would be insufficient as a means of verification.

Think of any of the optical illusions you've seen where two images become something completely different when combined at the correct angle. Looking at either of the component images in isolation, or removed from the 3D placement that creates the illusion, nothing would look amiss. But, view the model as it's "intended" (by the malicious designer) to be viewed, and "Oh, there's the genitalia." (To pluck a low-hanging fruit of an example.)

While that is a risk, I think to a certain extent its one we have to live with, and should just, you know, delete/relabel inappropriate media when we find it. Vandalism both subtle and otherwise has always been a threat, if we deemed it an unacceptable threat we wouldn't be able to have a wiki website at all. [But ultimately, i think this is a discussion best done on wiki ensuring the admins who would have to deal with the situation fully participate]

I understand the risk of having something inappropriate, but I think the risk is really low and whenever a wikimedian or commonist find it it would be deleted or solved. I have been looking to the examples given in the description and it would be something really valuable to have at Wikipedia articles.

To move the discussion on... is there any non-social blocker to do this?

In T246901#5971308, @Theklan wrote:

I understand the risk of having something inappropriate, but I think the risk is really low and whenever a wikimedian or commonist find it it would be deleted or solved. I have been looking to the examples given in the description and it would be something really valuable to have at Wikipedia articles.

To move the discussion on... is there any non-social blocker to do this?

@Theklan thanks, I think a couple of things I'd like to do before the discussion

• Outline exactly the process for implementing this after the community discussion, what work work would need to be done and roughly who would be capable of doing it and create phabcricator tasks for them

And for the discussion

• Plan the structure of the discussion and what will be explained and what will be asked
• Collate information on the educational impact of full colour 3D models
• Collate a fuller list of good quality 3d model sources

Anything else for the discussion you think is needed?

Thanks again

In T246901#5963328, @Multichill wrote:

I can't find it right now, but I recall having a list of requirements for new file formats to be support. Top of my head:

That list would be https://www.mediawiki.org/wiki/Manual:Adding_support_for_new_filetypes

@TheDJ thanks very much :)

@Keegan do you know what kinds of programming skills/languages would be needed to implement this?

Knowledge of that file format; 3D extension uses Javascript and PHP.

@Aklapper thanks very much

In T246901#5974963, @Mrjohncummings wrote:

@Keegan do you know what kinds of programming skills/languages would be needed to implement this?

If you (or anyone) is actually interested in doing the technical work for this, feel free to reach out with any questions. I'm not offering to do it, but I am offering to answer any questions about how code base works, if anyone gets "stuck" trying to do it.

@Bawolff thats very helpful, thanks very much, is there anything you could add to the task description that would help people to get started/not get stuck as easily?

@brion worked on glTF support a while ago. Looks to me like it got quite far, maybe possible to rebase it and take a stab at it.

https://gerrit.wikimedia.org/r/c/3d2png/+/415495

@brion could you describe what the issues you ran into were and how people could help?

Thanks

As a note of support, I'd very much love to see support for glTF. There are so many amazing full colour models of NASA spacecraft, such as https://mars.nasa.gov/resources/24584/curiosity-rover-3d-model/

I have just participated in the Hack4OpenGLAM Hackathon, part of the Creative Commons Global Summit 2020. One of the datasets available was a set of 1,000+ CC0-licensed 3D models of cultural heritage available on Sketchfab.

So I dug into this content for a week, and my conclusion is: 3D is certainly booming in cultural heritage, and there's so much fantastic content out there now. It has grown so much in the past few years.
But also, many models are only interesting with textures, unfortunately. This one is a great example - it's the glazing on the dish that makes it worth looking at.

You can see a summary of what I did at the hackathon here: https://commons.wikimedia.org/wiki/File:3D_models_from_Sketchfab_and_Wikimedia%E2%80%99s_structured_data_-_-Hack4OpenGLAM_at_CC_Summit_2020.pdf

FYI, in response to my hackathon project I was approached on Twitter by developers of the Modelviewer software, who suggested it as a tool to make integration of textured 3D models in currently widely supported file formats very easy for us.

I've created a separate ticket for that proposal: T266379: Use Modelviewer as 3D viewer for Wikimedia projects

In T246901#6575897, @Spinster wrote:

FYI, in response to my hackathon project I was approached on Twitter by developers of the Modelviewer software, who suggested it as a tool to make integration of textured 3D models in currently widely supported file formats very easy for us.

I've created a separate ticket for that proposal: T266379: Use Modelviewer as 3D viewer for Wikimedia projects

This would be really amazing, any idea how the process would work to use existing software?

@Spinster : "most models are only interesting with textures."

I could not agree more! Thank you for sharing your hackerthon summary and also looking into this. I suspect that one we have the ability to showcase textured 3D objects it will become a very important component of many Wikipedia articles.

For dynamic media like videos and 3D files, two different pieces of software are required. One is the thumbnailer, which turns the dynamic media into a static image. That's currently done by rTDTP 3d2png. The second is the client-side dynamic viewer, which allows users to view and interact with the dynamic content. That's currently implemented by Extension:3D as a part of MediaViewer. Both depend on https://threejs.org to do the actual heavy lifting of rendering 3D files with WebGL. Using the same library in both applications is important to reduce the visual difference between the static and dynamic versions and so that editors have a stable platform to work with.

As far as the process for implementing a third-party-developed viewer, the most recent example would probably be TimedMediaHandler's switch from Kaltura to VideoJS: T100106: Replace Kaltura player with Video.js. Extending MediaViewer like the current solution is likely to be the best option, if it's decided that switching renderers is a good idea. That is, however, likely to be more work than adapting the existing viewer.

@John_Cummings @Spinster I'm keen to see how this is coming. Is there an on-wiki discussion about it?

@Sj I'm sorry I missed this before, I don't think so, what would be the goal of having an on wiki discussion about it?

What would be a good path to achieving this? Who might possibly be interested in working on this and what would be the process to implement it?

From the GLAM mailing list [1]:

"Last week, the Wikimedia Foundation shared its draft annual plan on Meta and invited input via open calls in different time zones. You might have noticed that Maryana recognized that Commons is in desperate need of repair and made it a priority for the year ahead.

"There will be two calls dedicated to Commons:

"Wikimedia Commons 1 on Tuesday 26 April at 10:00 to 11:30 UTC

"Wikimedia Commons 2 on Thursday 28 April at 16:00 to 17:30 UTC"

Maybe this could be raised on one or other of those calls?

[1] https://lists.wikimedia.org/hyperkitty/list/glam@lists.wikimedia.org/thread/EJ262OLQUPBFL2HNOINYIGX5X65QAWJQ/

Just a quick heads up. I am seriously considering putting in a presentation application to talk about this subject (the need for 3D support on Commons/Wikipedia) at this year's Wikimania. I just wanted to know if anyone here is interested in joining me on that application? I feel even more strongly now on this issue than I did back in 2020 when I first commented on this issue.

Hello! I will be in Wikimania, I have other presentation plans, but I can help you with this, if there's appetite.

In T246901#8692485, @Theklan wrote:

Hello! I will be in Wikimania, I have other presentation plans, but I can help you with this, if there's appetite.

Thanks Theklan, that would be most welcome. Would you like to chat some time about it? The presentation plan at this stage is really very simple; here is the need for 3D support on Wiki and how it can be used, here are the advancements in displaying 3D objects else where, lets give developers support to work on this, Q&A and discussions.

I´m also interested in this and would like to help, I have examples of how could be used.
I would also like to chat with you.

@Discott you can contact me via Telegram, for sure. I'm thinking of people working on GLAM also. @GabrielLucas welcome!

I created this telegram group to talk about it
https://t.me/+tMgoJxx8D7I5NTQ8

@Discott @GabrielLucas @Theklan @John_Cummings nice to see! I'd be glad to be part of a workshop on supporting new formats, the overarching needs / costs / reasons-for-stalling are quite similar. And I think the best approach to realize this [since there is a necessary WM-core bottleneck] is to define a time-delimited project to identify and add a range of formats [and then update the workflow for doing so, ping interested people, &c].

Otherwise each subgroup wastes hundreds of hours working through the same bits of red tape.