Page MenuHomePhabricator

Explore moving the Panoviewer gadget/Toolforge tool into production
Open, Needs TriagePublic

Description

The work to create this tool by @dschwen is impressive, and got more so during the Wikimania 2016 hackathon with progressive HD download allowing for more performant use. See T105789: A new panoramic viewer for commons for more details.

The tool uses the Tool Labs grid for job execution and a library for tile manufacturer; theoretically it could be made into a real extension, with the tile manufactory put onto the Services cluster. This would allow it to be deployed everywhere, not just on Commmons, and integrated into MediaViewer officially rather than through a hack ;-) Maybe?

Event Timeline

Sounds good, what do you need from me?

If this gets integrated into the Mediaviewer we need to get additional data from the API, namely if the image really is a panorama (we'd need test if a specific template is used on the page, and in the case of non-spherical panos we need to fetch a template parameter for the location of the horizon line).

Currently the panoviewer only supports full spherical panoramics in equirectangular projection (which is exactly what "photospheres" are). Adding support for incomplete panos (either reduced vertical or horizontal FOV) should be straight forward though.

The tool consists of multiple parts:

  1. An HTML page (into which you pass the pano file name as a URL fragment (#). The page contains some Javascript that triggers the following components (this could all be in the Mediaviewer JS code in the future)
  2. A php script that returns a JSON config for the Pannellum WWebGL based panorama viewer. This script (https://github.com/Toollabs/panoviewer/blob/master/public_html/config.php) fetches images from commons (to get around origin policies), spawns the preprocessing on the grid and returns an intermediate resolution preview (JSON config). It also has a function used for polling if the grid job is done. (the JS in the HTML page under 1. is the driver for the polling)
  3. Pannellum (https://pannellum.org/) is the actual WebGL based viewer. It is great!
  4. A wrapper script that wraps pannellum's python script, that calls Nona and VIPS to reproject and tile high resolution panoramics (I submit thus using jsub from config.php)

I just had a long chat with @dschwen . It seems the best way to create image tiles would be to adapt Tilerator to convert large images into tiles, cache them into a tile storage (either SSDs or spinning rust), possibly relying on Cassandra's ability to distribute the storage, and serve them via Kartotherian, the same way as we serve maps. Very few backend modifications would be needed for this, other than to figure out where we want to actually store it, having enough capacity, storage configuration, etc.

Big unknown is if Mapnik can handle the required re-projection, or if we will need to have an external tool to do additional reprojection steps.

What we need is called a "gnomonic" projection. Here's an example: https://mycodingwrongs.wordpress.com/2010/07/24/reprojecting-blue-marble/

This might be totally doable!!!

Added a Mapnik issue, will see if anyone replies.

Jdforrester-WMF moved this task from Untriaged to Next up on the Multimedia board.

I now have:

  • changes to our xmp-reader, which read the relevant metadata (still needs test)
  • changes to the thumbnail renderer, to check the metadata for panoramas and to add a data-projection-type attribute if appropriate (T151451)
  • a userscript to replace images annotated with data-projection-type, with an iframed https://tools.wmflabs.org/panoviewer/ (if i disable crossorigin checks in my browser)

https://youtu.be/JAVdC2PRS00

Alternative script library that has provisions for iOS panorama's https://github.com/naver/egjs-view360

I've now got a PanoViewer extension with Pannellum, and using the new XMP and iPhone metadata.

TODO:

  • Still uses AR of original image to render thumbnail. Less than ideal usually.
  • Panorama/Cylindrical projection
  • Polar projection ?
  • Tile rendering/pannellum multires
  • Support for MMV
  • Support for projection metadata inside of CommonsData
  • Annotations metadata inside of CommonsData
  • Image syntax parameter to enabled/disable viewer
  • an iframe mode

Still considering switching to egjs-view360, as that does provide support for Panoramas. Unfortunately egjs doesn't support multires tile rendering like pannellum does. Nothing seems to support polar projection. Can also consider to implement panorama support using another (simpler) library of course..
The biggest problem with Panoramas is that many are phone panoramas (without full metadata), and you basically need to infer fov settings.. It seems facebook does this based on camera model + camera settings + file dimensions... but it doesn't seem anyone has made an open source version of this yet.

Also pondering about how to deal with panorama thumbnails inside articles. When to have small and when to have large ones...

Just as a quick comment, I think multires tiles are absolutely necessary for the stitched panos that we have (>100 Megapixel). I would suggest not even thinking about a solution that does not permit this.

Aklapper lowered the priority of this task from High to Lowest.Sep 13 2021, 5:07 AM
Aklapper added a project: Tools.

"High" priority for five years 🡒 lowering. Also this task has zero active project tags due to WMF's "change management".

@Aklapper, this task ranked highly in the most recent community wishlist and the Community Tech team's subsequent prioritization process, so I'm not sure describing it as "lowest" is accurate. I certainly hope it's not considered that unimportant.

@Sdkb: Priority should reflect reality and does not cause it. If someone actively plans to work on this (it would be welcome), then they are welcome to increase priority.

Sdkb raised the priority of this task from Lowest to Needs Triage.Sep 24 2021, 9:58 PM
Sdkb added a project: Community-Tech.

My point is that the Community Tech team does plan to work on this, at least from what they've communicated. I've added it to their group and removed the priority so that they can assess it themselves.

MusikAnimal added a subscriber: MusikAnimal.

My point is that the Community Tech team does plan to work on this, at least from what they've communicated. I've added it to their group and removed the priority so that they can assess it themselves.

We discussed this with the team, and no one was able to confirm we explicitly said we are going to take on this project. We apologize if anyone was misled. Yes, the wish performed better this year than in previous years, and we certainly we like to work on it (as with any popular wish), but we can't commit to that until current priorities are completed. In my opinion, unfortunately it seems unlikely we'd get to it before the next survey starts in January. If that turns out to be true, we ask that the wish be re-proposed, and after the survey we will prioritize it accordingly.

For future reference, we typically do not add the Community-Tech tag to wishlist tasks until we are sure we can commit to them. I have updated our project description accordingly. Thanks for your understanding,

Thanks for the update/clarification, @MusikAnimal. For reference, by "ranked highly in the most recent community wishlist and the Community Tech team's subsequent prioritization process" I was referring to this report. I certainly understand that the resources of the team are extremely limited, and I'm not saying this particular task should displace others, but it remains very frustrating that 91 different editors flagging a task as important is still not enough to get the WMF to allocate any resources at all to that task.

For reference, by "ranked highly in the most recent community wishlist and the Community Tech team's subsequent prioritization process" I was referring to this report.

Glad you saw that :) If it wasn't clear, that's not us committing to everything listed there, rather it's our own prioritized list from which we work from the top-bottom. Sorry for the confusion. If you have any further questions about this process or the wishlist in general, we'd love to engage, but we should discuss over on Meta to avoid more unrelated noise here. Thanks, and apologies again if we misled you or anyone else!

A point that recently came up on en-WP (kudos to Magnolia677) is that the version of the panorama viewer does not include an easily noticeable link to the Commons file page, only the enlarge button in the caption that's very rarely used. This makes it more difficult to see information (like description/date) of files and to access tools like the download button. A redesign of the rendering gadget should address this.

taavi renamed this task from Explore moving the Panoviewer gadget/Tool Labs tool into production to Explore moving the Panoviewer gadget/Toolforge tool into production.Oct 21 2021, 5:10 AM

https://gitlab.wikimedia.org/hartman/panoviewer

I also found a new library worth exploring called panolens.js

I know this may be beyond the scope of just this one task, but it would be nice to be able to support clicking on hotspots in the 360 panorama/photosphere to navigate to other ones (ie. virtual tour mode). Pannellum supports this quite nicely already. It would be pretty cool to see Commons community perhaps stitching multiple spheres for a location (Wiki Loves Monuments comes to mind) to provide a more immersive and interactive experience.

https://pannellum.org/documentation/examples/tour/