Page MenuHomePhabricator

Have search recognise STL files as a new kind of media file ('type:3d' or whatever)
Closed, ResolvedPublic

Description

Add something to the ElasticSearch indexes, presumably?

Event Timeline

Change 336454 had a related patch set uploaded (by MarkTraceur):
Add 3D filetype for STL, AMF files

https://gerrit.wikimedia.org/r/336454

Reedy renamed this task from Have search recognise AMF and STL files as a new kind of media file ('type:3d' or whatever) to Have search recognise STL files as a new kind of media file ('type:3d' or whatever).Feb 27 2017, 11:53 PM

Dear wonderful DBA s, could one of you briefly look at https://gerrit.wikimedia.org/r/#/c/336454/ (especially https://gerrit.wikimedia.org/r/#/c/336454/6/maintenance/archives/patch-add-3d.sql ) to see if it's OK from your POV? If so, we'll merge and then create a task for a production DB adjustment for this. Thanks!

I would have to test that I am right at the CR to provide a more accurate ETA. It will not be 100% trivial in any case because commons' heavy usage of the image table, but we will see how we can do it best.

dr0ptp4kt raised the priority of this task from Medium to High.May 31 2017, 5:14 PM
dr0ptp4kt moved this task from Untriaged to Next up on the Multimedia board.
dr0ptp4kt subscribed.

We'll be circling on this one soon.

@jcrespo we're interested in merging - would you be able to test so as to confirm your code review?

Merge- if it is slow or not, it has to be applied, the only difference is if it will take hours or weeks to apply- as long as no code depends on it existing, there is no problem, just follow https://www.mediawiki.org/wiki/Development_policy#Database_patches and https://wikitech.wikimedia.org/wiki/Schema_changes#Workflow_of_a_schema_change

@jcrespo since this change is only really necessary on the beta cluster right now, do we need to file a ticket for that specifically, or is it a separate process that we should do ourselves?

Also, it looks like the wikis we need to apply this to (i.e. 'wikis where uploads are enabled') may be difficult to list explicitly based on the current WMF config - is there a better way to specify that?

I have no power over there... on beta. That is supposedly to be used for people to test their changes before anything reaches production. In the places were beta testing is not adequate, testwiki and sister projects (on production) can be used- which are all normally very easy to apply a schema change to.

The workflow I mentioned is for wmf production deployment (that means enwiki and the other 900 wikis with well-known domains.

"may be difficult to list explicitly based on the current WMF config" If you do not know the list, I will not be able to apply it :-) Looking at your change, given that it alters tables.sql on core, normally that would mean you have to reference all.dblist, unless something very very specific is needed (all -test because test has been done; only enwiki becase there was a bug that only applies to wikis older than 17 years, etc.).

On the above guide:

Note that this means your schema change should be optional in code - for wikimedia deployments, it is expected that every wiki with the relevant database table(s) will have the schema change applied to them. If you need different schema for different wikis, then apply the change using an extension and creating new tables dependent on that extension.

That means that if the code is deployed, it should be requested to be applied to all wikis if it is a core change, but that the code should not exect the change to be available from the begining, a schema change takes, in the optimal case, 1 week to propagate; in the worse case, several years.

Change 336454 merged by jenkins-bot:
[mediawiki/core@master] Add 3D filetype for STL files

https://gerrit.wikimedia.org/r/336454

Change 336454 merged by jenkins-bot:
[mediawiki/core@master] Add 3D filetype for STL files

https://gerrit.wikimedia.org/r/336454

This needs to update other db backends as well, not just MySQL.

This patch caused the Travis CI build for wikimedia/mediawiki to fail on Postgres, due to the ENUM being incomplete and failing Upload and Search unit tests.

Change 366563 had a related patch set uploaded (by Matthias Mullie; owner: Matthias Mullie):
[mediawiki/core@master] Follow-up to '3D' file type addition

https://gerrit.wikimedia.org/r/366563

dr0ptp4kt claimed this task.

This has been verified. Bypassing manual QA.

Krinkle reopened this task as Open.EditedJul 24 2017, 11:39 PM

Re-opening until "the build" is no longer broken. The patch for this task added a MySQL schema change to MediaWiki without updating the other schemas, which creates a fatal error for all non-MySQL installations. (Including Sqlite and Postgres.)

Change 366563 merged by jenkins-bot:
[mediawiki/core@master] Follow-up to '3D' file type addition

https://gerrit.wikimedia.org/r/366563

Thanks. Confirmed on Travis CI.