Page MenuHomePhabricator

Develop a Media JSON API
Closed, ResolvedPublic

Description

Develop a JSON API that returns all media for a page.

This includes: page images, audio files, and media files.

See also: https://www.mediawiki.org/wiki/Specs/HTML/1.6.0

General Logic


Commons Info

  • For each item, we should be querying commons API to get licensing, captions, file page url, author, mime-type etc…

Open question: For mime type: is it possible to have different mime types per item? Do we need to have sub dictionaries for mime types?
For example: for images is it possible that the thumb and original are 2 different mime types?

Wikipedia Captions (Or original Wiki caption)

  • In addition to the commons caption, we should also return the Wikipedia caption that is in the page where the image is located.

Section Information

  • We should return the ID of the section that contains each item.

Media Type

  • Each item in the array should be given a type to denote what kind of object it is. Proposed types: "Image", "Video", "Audio"

Extended metadata

  • extmetadata items of interest appear in the top level

Misc

  • Return page_count for paged file types (PDF, TIFF, DjVu...)

Type specific logic and metadata


Images

This is essentially replacing the client side logic of:

  • parsing out image urls
  • rewriting urls to get both original and thumb sizes
  • Identifying images that are too small for a gallery
  • The API should return an array of image objects (with thumb and full sizes) like in other MCS APIs with the other information included.
Images that are too small for the gallery
  • Stop filtering images based on file type (SVG, PNG).
  • Start filtering image based on size on the page (reject if width or height < 64px).
  • Add mw-gallery markup with grouping (T182330)
  • Add filtering for noviewer, metadata classes

Audio/Video

  • We should return duration.
  • If start and end times are specified, we should return them
  • Poster (image) for the video shows the time of the image
  • Add captioning info (exposed via <track> elements? (created T185263 to discuss)

Open question: anything else?

Audio special cases
  • We should mark pronunciations as a special kind of media file
  • We should mark Spoken Wikipedia audio files as a special kind of file

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 394402 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Add diff test for the media endpoint

https://gerrit.wikimedia.org/r/394402

Change 392728 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Add audio files to response

https://gerrit.wikimedia.org/r/392728

Change 392868 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Add on-wiki captions to media response

https://gerrit.wikimedia.org/r/392868

Change 392907 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Add thumb_mime and duration to media output

https://gerrit.wikimedia.org/r/392907

Change 392979 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: move extmetadata properties of interest into the top level

https://gerrit.wikimedia.org/r/392979

Change 393806 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Report image properties in standard image objects

https://gerrit.wikimedia.org/r/393806

Change 394574 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Add thumb_time to video items, if it exists

https://gerrit.wikimedia.org/r/394574

Change 394609 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] WIP: Add media lib unit tests

https://gerrit.wikimedia.org/r/394609

Change 394610 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Hygiene: Add media selector unit tests

https://gerrit.wikimedia.org/r/394610

Change 394609 abandoned by Mholloway:
WIP: Add media lib unit tests

Reason:
oops

https://gerrit.wikimedia.org/r/394609

Change 394641 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: exclude 'width' and 'height' properties from audio items

https://gerrit.wikimedia.org/r/394641

Change 394598 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Mark spoken WP files with audio_type: spoken

https://gerrit.wikimedia.org/r/394598

Change 394641 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: exclude 'width' and 'height' properties from audio items

https://gerrit.wikimedia.org/r/394641

Change 394652 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Unit testing for metadata and derivatives parsing

https://gerrit.wikimedia.org/r/394652

Change 394610 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Hygiene: Add media selector unit tests

https://gerrit.wikimedia.org/r/394610

Mholloway updated the task description. (Show Details)Dec 1 2017, 9:32 PM
Mholloway updated the task description. (Show Details)

Change 394719 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Add pronunciation audio files

https://gerrit.wikimedia.org/r/394719

Mholloway updated the task description. (Show Details)Dec 1 2017, 10:41 PM

Change 394652 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Unit testing for metadata and derivatives parsing

https://gerrit.wikimedia.org/r/394652

Change 394719 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Add pronunciation audio files

https://gerrit.wikimedia.org/r/394719

Mholloway updated the task description. (Show Details)Dec 4 2017, 7:16 PM
Mholloway updated the task description. (Show Details)Dec 5 2017, 4:41 PM
Mholloway added a subtask: Restricted Task.Dec 5 2017, 5:36 PM

Change 395585 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media endpoint output tweaks, pt. 1

https://gerrit.wikimedia.org/r/395585

Change 395599 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media endpoint output tweaks, pt. 2: derivatives

https://gerrit.wikimedia.org/r/395599

Change 395643 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Add page count for paged items (PDF, TIFF)

https://gerrit.wikimedia.org/r/395643

Change 395653 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Exclude .noviewer and .metadata items

https://gerrit.wikimedia.org/r/395653

Change 395585 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media endpoint output tweaks, pt. 1

https://gerrit.wikimedia.org/r/395585

Change 395599 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media endpoint output tweaks, pt. 2: derivatives

https://gerrit.wikimedia.org/r/395599

Change 395643 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Add page count for paged items (PDF, TIFF, DjVu, etc.)

https://gerrit.wikimedia.org/r/395643

Change 395653 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Exclude .noviewer and .metadata items

https://gerrit.wikimedia.org/r/395653

Change 395849 had a related patch set uploaded (by BearND; owner: BearND):
[mediawiki/services/mobileapps@master] Version number for the media endpoint

https://gerrit.wikimedia.org/r/395849

Change 395849 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Version number for the media endpoint

https://gerrit.wikimedia.org/r/395849

Mholloway updated the task description. (Show Details)Dec 7 2017, 2:52 PM
Mholloway updated the task description. (Show Details)Dec 7 2017, 3:03 PM

Change 396031 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Update selector blacklist

https://gerrit.wikimedia.org/r/396031

Mholloway updated the task description. (Show Details)Dec 7 2017, 3:58 PM
Mholloway updated the task description. (Show Details)

Change 396031 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Update selector blacklist

https://gerrit.wikimedia.org/r/396031

Change 396085 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Add section numbers

https://gerrit.wikimedia.org/r/396085

Change 396420 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Remove unwanted selectors from the blacklist

https://gerrit.wikimedia.org/r/396420

Change 396085 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Add section ID numbers

https://gerrit.wikimedia.org/r/396085

Change 396420 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Remove unwanted selectors from the blacklist

https://gerrit.wikimedia.org/r/396420

Mholloway updated the task description. (Show Details)Dec 11 2017, 1:40 PM
Mholloway updated the task description. (Show Details)
Mholloway updated the task description. (Show Details)

Change 397841 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Add video derivative sizes

https://gerrit.wikimedia.org/r/397841

Mholloway updated the task description. (Show Details)Dec 12 2017, 3:22 PM
Mholloway updated the task description. (Show Details)Dec 12 2017, 4:34 PM

Change 397994 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Mark items in MW galleries with gallery IDs

https://gerrit.wikimedia.org/r/397994

Mholloway updated the task description. (Show Details)Dec 13 2017, 2:16 AM

Change 397994 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Mark items in MW galleries with gallery IDs

https://gerrit.wikimedia.org/r/397994

Change 397841 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Add video derivative sizes

https://gerrit.wikimedia.org/r/397841

Stashbot added a subscriber: Stashbot.

Mentioned in SAL (#wikimedia-operations) [2018-01-04T18:52:14Z] <bsitzmann@tin> Started deploy [mobileapps/deploy@8bcffa9]: Update mobileapps to a4ba9fd (T182330 T177430 T170690 T182652 T184198)

Mentioned in SAL (#wikimedia-operations) [2018-01-04T18:58:14Z] <bsitzmann@tin> Finished deploy [mobileapps/deploy@8bcffa9]: Update mobileapps to a4ba9fd (T182330 T177430 T170690 T182652 T184198) (duration: 06m 01s)

Mholloway updated the task description. (Show Details)Jan 18 2018, 4:26 PM

Change 405009 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Stop filtering PNGs and SVGs by type

https://gerrit.wikimedia.org/r/405009

Change 405010 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media: Filter images based on size on the page

https://gerrit.wikimedia.org/r/405010

Change 405009 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Stop filtering PNGs and SVGs by type

https://gerrit.wikimedia.org/r/405009

Change 405010 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media: Filter images based on size on the page

https://gerrit.wikimedia.org/r/405010

Mentioned in SAL (#wikimedia-operations) [2018-01-18T18:47:35Z] <bsitzmann@tin> Finished deploy [mobileapps/deploy@669fb5b]: Update mobileapps to 2690899 (T184328 T184557 T177007 T184669 T177430 T185050) (duration: 07m 03s)

Mholloway updated the task description. (Show Details)Jan 18 2018, 10:11 PM

I've updated the example output at https://www.mediawiki.org/wiki/Page_Content_Service/Media with the latest (sorry, I'll have to go back and <nowiki> it up another time). With that I think this is essentially ready to go, perhaps pending a meeting with the client devs to review the output.

I don't think there is need for <nowiki>. I just used <syntaxhighlight lang="json"> for the whole output.

Mholloway updated the task description. (Show Details)Feb 26 2018, 7:35 PM

Change 417008 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media endpoint response tweaks

https://gerrit.wikimedia.org/r/417008

Change 417008 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Media endpoint response tweaks

https://gerrit.wikimedia.org/r/417008

Change 419319 had a related patch set uploaded (by Mholloway; owner: Mholloway):
[mediawiki/services/mobileapps@master] Media endpoint performance tweaks

https://gerrit.wikimedia.org/r/419319

Change 419319 merged by Mholloway:
[mediawiki/services/mobileapps@master] Media endpoint performance tweaks

https://gerrit.wikimedia.org/r/419319

@Mholloway We still need to do some performance tweaks.

Mholloway closed subtask Restricted Task as Resolved.Apr 9 2018, 5:39 PM