Adding T103421 as dep; proper way to check is to actually look at the header packets and check what they contain, which File_Ogg package can do. We need to move our copy out of TMH and use it via Composer in a shared manner.

I believe the problem here is partially that we are not guaranteed to have skeleton on original files right ? Making detection more involved/expensive for those files ?

Skeleton track, if present, can list the types of the various streams, but yes it's usually not there for audio and not always there for video depending on which software was used to convert. But parsing the skeleton track isn't trivial either -- it's embedded in the Ogg stream multiplexing too -- so we may as well just check the header packets of each stream, which is what File_Ogg does.

If you dive into it you might be horrified to see that it also does substring matches... ;) but it's checking the actual initial header packets, not just 'whatever is in the first 255 bytes' so shouldn't fall prey to false positives (like the hack to avoid matching on comments mentioning ffmpeg2theora)

The whole concept of ogg horrifies me and always has :)

Right, so I think that part of the problem here, is that this information is simply at the wrong place.

1: We infer the media-type based on the mime-type
2: The mime-type is what controls the handler we use and thus reflects the file's mime-type (application/ogg)
3: In the 'frontend' however, we often are interested in the an alias mime type (audio/vorbis instead of application/ogg )
4: The alias mime-type is based on the codecs and type of tracks used inside the file
5: The alias mime-type can thus only be accurately given by the file handler.
6: The proper media-type can thus only really be determined by the handler.
7: We have a 'hack' in MimeAnalyzer that allows to find an 'alias' based on the file extension, but not all files use separate file extensions for these alias mime types.
8: Our Swift filebackend (which serves the frontend) ignores most of this, and does it's own mime-type detection. T131012: uploaded SVGs from 2015/2016 without XML prolog (<?xml ... ?>) served with Content-Type text/html from upload.wikimedia.org

Questions:
1: What is the intended purpose of media_type ? We need to clarify this.
2: Do we actually need this information in the database ? Maybe it's enough if the file handler can provide it ? Maybe it should be in a separate table for caching purposes ? This is also a question raised in T589: RFC: image and oldimage tables.
3: should we make the Handler (through the File interface) the canonical information provider for mime and media-type information instead of MimeAnalyzer filling the database with guesses ?
4: Or are we going to create complex integrations directly in MimeAnalyzer with the handlers and/or their format libraries, to provide directly the desired information to the Analyzer ? That would add a lot of extra complexity to this relatively simple class.
5: How does this mix with the more advanced mime inspection that the file commandline utility does ?

In T155320#2949427, @brion wrote:

If you dive into it you might be horrified to see that it also does substring matches... ;) but it's checking the actual initial header packets, not just 'whatever is in the first 255 bytes' so shouldn't fall prey to false positives (like the hack to avoid matching on comments mentioning ffmpeg2theora)

If you mean a scan for the string "OggS", note that the Ogg spec has a resynchronisation feature which requires scanning the data for page headers, identified by that string, which is called the capture pattern. "The bitstream is captured (or recaptured) by looking for the beginning of a page, specifically the capture pattern. Once the capture pattern is found, the decoder verifies page sync and integrity by computing and comparing the checksum. At that point, the decoder can extract the packets themselves." https://www.xiph.org/vorbis/doc/framing.html

This mime-type thing keeps bothering me. I was thinking the following:

1. make mime_type the actual 'frontend' desired mimetype of a specific file.
2. add a new column filehandler_id
3. add filehandler_id's to our FileHandlers
4. add a scan method to file handlers to detect magic bytes, return true or false depending on if this handler can handle this file.
5. when ingesting files, scan with each file handler to find the right file handler
6. add a 'getMimeType' to file handlers to return the correct mime type for a file.
7. register the file handler_id into the database for fast instantiation of the file handler, store the mime-type alias in the database for fast serving of the correct mime_type.

Maybe simply base the whole MediaHandler off of ContentHandler concepts ?

Could I suggest that audio files are categorized as audio/ogg and video files as video/ogg instead of both of them application/ogg? All those media types are valid ones: https://en.wikipedia.org/wiki/Ogg But it would be really nice if it was possible from the media type to know if it is audio or video.

From my observations looking at imageinfo metadata available for these files (as available in image Mediawiki table), I think an easy fix for many (if not all) of these files would be to simply look at existing imageinfo metadata: if a file is application/ogg and has width and height, mark it as video, if width and height is 0, mark it as audio. Similar for audio/webm and video/webm. So this could be fixed now for existing files with a simple script going over the image table. While we wait for the perfect fix for new files.

I filled T301807 for two MPG files which are misclassified.

In T155320#7643181, @Mitar wrote:

Could I suggest that audio files are categorized as audio/ogg and video files as video/ogg instead of both of them application/ogg? All those media types are valid ones: https://en.wikipedia.org/wiki/Ogg But it would be really nice if it was possible from the media type to know if it is audio or video.

Unfortunately the current way the media engine of MediaWiki is written can not do this right now. Its something that should be fixed, but as we all know investment into the media capabilities of MediaWiki is considerably poor.

I see now that 157 OGG audio files at rowiki are identified as video, see https://ro.wikipedia.org/wiki/Special:Search/file:_local:_filetype:video_fileh:0.