Page MenuHomePhabricator

Explore moving the Panoviewer gadget/Tool Labs tool into production
Open, HighPublic

Description

The work to create this tool by @dschwen is impressive, and got more so during the Wikimania 2016 hackathon with progressive HD download allowing for more performant use. See T105789: A new panoramic viewer for commons for more details.

The tool uses the Tool Labs grid for job execution and a library for tile manufacturer; theoretically it could be made into a real extension, with the tile manufactory put onto the Services cluster. This would allow it to be deployed everywhere, not just on Commmons, and integrated into MediaViewer officially rather than through a hack ;-) Maybe?

Event Timeline

Restricted Application added subscribers: Zppix, Matanya, Aklapper. · View Herald TranscriptJun 29 2016, 11:41 AM

Sounds good, what do you need from me?

If this gets integrated into the Mediaviewer we need to get additional data from the API, namely if the image really is a panorama (we'd need test if a specific template is used on the page, and in the case of non-spherical panos we need to fetch a template parameter for the location of the horizon line).

Currently the panoviewer only supports full spherical panoramics in equirectangular projection (which is exactly what "photospheres" are). Adding support for incomplete panos (either reduced vertical or horizontal FOV) should be straight forward though.

The tool consists of multiple parts:

  1. An HTML page (into which you pass the pano file name as a URL fragment (#). The page contains some Javascript that triggers the following components (this could all be in the Mediaviewer JS code in the future)
  2. A php script that returns a JSON config for the Pannellum WWebGL based panorama viewer. This script (https://github.com/Toollabs/panoviewer/blob/master/public_html/config.php) fetches images from commons (to get around origin policies), spawns the preprocessing on the grid and returns an intermediate resolution preview (JSON config). It also has a function used for polling if the grid job is done. (the JS in the HTML page under 1. is the driver for the polling)
  3. Pannellum (https://pannellum.org/) is the actual WebGL based viewer. It is great!
  4. A wrapper script that wraps pannellum's python script, that calls Nona and VIPS to reproject and tile high resolution panoramics (I submit thus using jsub from config.php)
Yurik added a subscriber: Yurik.Jun 30 2016, 1:52 AM

I just had a long chat with @dschwen . It seems the best way to create image tiles would be to adapt Tilerator to convert large images into tiles, cache them into a tile storage (either SSDs or spinning rust), possibly relying on Cassandra's ability to distribute the storage, and serve them via Kartotherian, the same way as we serve maps. Very few backend modifications would be needed for this, other than to figure out where we want to actually store it, having enough capacity, storage configuration, etc.

Big unknown is if Mapnik can handle the required re-projection, or if we will need to have an external tool to do additional reprojection steps.

What we need is called a "gnomonic" projection. Here's an example: https://mycodingwrongs.wordpress.com/2010/07/24/reprojecting-blue-marble/

This might be totally doable!!!

Yurik added a comment.Jun 30 2016, 5:25 AM

Added a Mapnik issue, will see if anyone replies.

Jdforrester-WMF triaged this task as High priority.Jul 20 2016, 6:43 PM
Jdforrester-WMF moved this task from Untriaged to Next up on the Multimedia board.
Pchelolo moved this task from Backlog to watching on the Services board.Oct 12 2016, 5:49 PM
Pchelolo edited projects, added Services (watching); removed Services.
mxn added a subscriber: mxn.Nov 10 2018, 10:10 PM
TheDJ added a subscriber: TheDJ.EditedAug 20 2019, 10:07 PM

I now have:

  • changes to our xmp-reader, which read the relevant metadata (still needs test)
  • changes to the thumbnail renderer, to check the metadata for panoramas and to add a data-projection-type attribute if appropriate (T151451)
  • a userscript to replace images annotated with data-projection-type, with an iframed https://tools.wmflabs.org/panoviewer/ (if i disable crossorigin checks in my browser)

https://youtu.be/JAVdC2PRS00

TheDJ added a comment.Aug 27 2019, 8:17 PM

Alternative script library that has provisions for iOS panorama's https://github.com/naver/egjs-view360

I've now got a PanoViewer extension with Pannellum, and using the new XMP and iPhone metadata.

TODO:

  • Still uses AR of original image to render thumbnail. Less than ideal usually.
  • Panorama/Cylindrical projection
  • Polar projection ?
  • Tile rendering/pannellum multires
  • Support for MMV
  • Support for projection metadata inside of CommonsData
  • Annotations metadata inside of CommonsData
  • Image syntax parameter to enabled/disable viewer
  • an iframe mode

Still considering switching to egjs-view360, as that does provide support for Panoramas. Unfortunately egjs doesn't support multires tile rendering like pannellum does. Nothing seems to support polar projection. Can also consider to implement panorama support using another (simpler) library of course..
The biggest problem with Panoramas is that many are phone panoramas (without full metadata), and you basically need to infer fov settings.. It seems facebook does this based on camera model + camera settings + file dimensions... but it doesn't seem anyone has made an open source version of this yet.

Also pondering about how to deal with panorama thumbnails inside articles. When to have small and when to have large ones...

Restricted Application added a subscriber: RhinosF1. · View Herald TranscriptDec 19 2019, 10:35 AM

Just as a quick comment, I think multires tiles are absolutely necessary for the stitched panos that we have (>100 Megapixel). I would suggest not even thinking about a solution that does not permit this.