Page MenuHomePhabricator

Images should have a projection and/or stereo attribute, reflected by HTML and/or api
Open, MediumPublic

Description

Much like animated images are detected to be animated and registered as such in the database, we should probably have this information for 3D, 360 and pano images. Currently such information is added by including a template on the File page, but this doesn't give you any usable information when transcluding, reusing or viewing in MMV etc.

This would simplify usage for:
T138933: Explore moving the Panoviewer gadget/Toolforge tool into production
T105789: A new panoramic viewer for commons
T70719: Tool to view spherical panorama photos (photospheres)

Event Timeline

MarkTraceur moved this task from Untriaged to Tracking on the Multimedia board.

Note to self, Apple uses EXIF CustomRendered == 6 (panorama)

Another note to self: once implemented, this requires an update to the cached metadata using maintenance/refreshMetadata --force

Note with list of more markers in various formats for stereoscopic content.
https://www.sview.ru/en/help/input/

  • Multiple streams (*.mpo, *.mkv, *.wmv). Two image / video streams of the same size are automatically detected as stereoscopic pair with Left view in the first stream.
  • JPEG image containing JPS marker (VRex extension). JPS marker defines stereoscopic format.
  • PNG image containing sTER chunk (Extensions to the PNG). Only parallel pair with views order can be defined by sTER indicator.
  • WMV metadata (*.wmv). The following metadata fields will be interpreted by sView:
    • StereoscopicLayout defining one of layouts (SideBySideRF, SideBySideLF, OverUnderLT, OverUnderRT)
    • StereoscopicHalfHeight and StereoscopicHalfWidth defining anamorphic video
    • StereoscopicHorizontalParallax defining parallax in pixels
  • MKV metadata (*.mkv, *mkv3d). Matroska specification includes dedicated fields defining stereoscopic layout (StereoMode). FFmpeg library converts this data in form of stream metadata with name STEREO_MODE (with values mono, right_left, left_right, bottom_top, top_bottom, row_interleaved_rl, row_interleaved_lr, block_lr, block_rl, anaglyph_cyan_red, anaglyph_green_magenta). Note that detection code in sView is generalized and specified metadata tags will be read from any video file (not only *.mkv).
  • h264 SEI messages (per frame side data, *.mp4, *.mkv). Some codecs stores stereoscopic identification data at every frame. This technically allows to switch from mono to stereo3d within the same stream. FFmpeg provides this information in form of AVStereo3D structure - thus sView will be able to read this information for all decoders in FFmpeg supporting this feature.
  • File extension (*.pns, *.jps). Image files with extensions *.pns (PNG file) and *.jps (JPEG file) will be interpreted as stereoscopic pair in Side-by-side format with Right view first (cross-eyed).

File name. sView supports the following name convention for stereoscopic format identification:
half-ou, -hou, -abq define anamorphic Over/Under pair
half-sbs, -hsbs, -lrq, -rlq define anamorphic Side-by-side pair
-ba, -ab define Over/Under pair
-sbs, -lr, -rl define Side-by-side pair

Aspects identified so far:

Definitions:

  • Horizontal Field of view (amount of degrees covering left to right)
  • Vertical Field of View (amount of degrees covering left to right)
  • Angular Field of View (still don't fully get this)
  • Heading/InitialViewHeading: horizontal direction (compass) that the camera entered at, or that the view should be initialised with.
  • Projection: The way the image is distorted to the image H-FoV/V-FoV
  • Stereoscopic: The way two angles are stored for 3D stereoscopic viewing
    • mono (no stereoscopy)
    • side by side (lr/rl)
    • top by bottom (tb/bt)
    • interlaced/row interleaved (rare)
    • anaglyph_cyan_red / anaglyph_green_magenta
    • multiple streams (also seems rather rare)
  • field of view: horizontal/vertical/direction
    • Standard
    • 360 (true 360, sphere's cubes etc)
    • Panorama shots (horizontal considerably wider than vertical)

Matroska considers rotation to be a projection as well.

TheDJ renamed this task from Images should have a projection attribute, reflected by HTML and/or api to Images should have a projection and/or stereo attribute, reflected by HTML and/or api.May 10 2022, 9:08 PM
TheDJ updated the task description. (Show Details)