If we did this, it would fulfil T70719: Tool to view spherical panorama photos (photospheres).

Sounds good, what do you need from me?

If this gets integrated into the Mediaviewer we need to get additional data from the API, namely if the image really is a panorama (we'd need test if a specific template is used on the page, and in the case of non-spherical panos we need to fetch a template parameter for the location of the horizon line).

Currently the panoviewer only supports full spherical panoramics in equirectangular projection (which is exactly what "photospheres" are). Adding support for incomplete panos (either reduced vertical or horizontal FOV) should be straight forward though.

The tool consists of multiple parts:

1. An HTML page (into which you pass the pano file name as a URL fragment (#). The page contains some Javascript that triggers the following components (this could all be in the Mediaviewer JS code in the future)
2. A php script that returns a JSON config for the Pannellum WWebGL based panorama viewer. This script (https://github.com/Toollabs/panoviewer/blob/master/public_html/config.php) fetches images from commons (to get around origin policies), spawns the preprocessing on the grid and returns an intermediate resolution preview (JSON config). It also has a function used for polling if the grid job is done. (the JS in the HTML page under 1. is the driver for the polling)
3. Pannellum (https://pannellum.org/) is the actual WebGL based viewer. It is great!
4. A wrapper script that wraps pannellum's python script, that calls Nona and VIPS to reproject and tile high resolution panoramics (I submit thus using jsub from config.php)

I just had a long chat with @dschwen . It seems the best way to create image tiles would be to adapt Tilerator to convert large images into tiles, cache them into a tile storage (either SSDs or spinning rust), possibly relying on Cassandra's ability to distribute the storage, and serve them via Kartotherian, the same way as we serve maps. Very few backend modifications would be needed for this, other than to figure out where we want to actually store it, having enough capacity, storage configuration, etc.

Big unknown is if Mapnik can handle the required re-projection, or if we will need to have an external tool to do additional reprojection steps.

What we need is called a "gnomonic" projection. Here's an example: https://mycodingwrongs.wordpress.com/2010/07/24/reprojecting-blue-marble/

This might be totally doable!!!

Added a Mapnik issue, will see if anyone replies.

I now have:

• changes to our xmp-reader, which read the relevant metadata (still needs test)
• changes to the thumbnail renderer, to check the metadata for panoramas and to add a data-projection-type attribute if appropriate (T151451)
• a userscript to replace images annotated with data-projection-type, with an iframed https://tools.wmflabs.org/panoviewer/ (if i disable crossorigin checks in my browser)

https://youtu.be/JAVdC2PRS00

Alternative script library that has provisions for iOS panorama's https://github.com/naver/egjs-view360

I've now got a PanoViewer extension with Pannellum, and using the new XMP and iPhone metadata.

TODO:

• Still uses AR of original image to render thumbnail. Less than ideal usually.
• Panorama/Cylindrical projection
• Polar projection ?
• Tile rendering/pannellum multires
• Support for MMV
• Support for projection metadata inside of CommonsData
• Annotations metadata inside of CommonsData
• Image syntax parameter to enabled/disable viewer
• an iframe mode

Still considering switching to egjs-view360, as that does provide support for Panoramas. Unfortunately egjs doesn't support multires tile rendering like pannellum does. Nothing seems to support polar projection. Can also consider to implement panorama support using another (simpler) library of course..
The biggest problem with Panoramas is that many are phone panoramas (without full metadata), and you basically need to infer fov settings.. It seems facebook does this based on camera model + camera settings + file dimensions... but it doesn't seem anyone has made an open source version of this yet.

Also pondering about how to deal with panorama thumbnails inside articles. When to have small and when to have large ones...

Just as a quick comment, I think multires tiles are absolutely necessary for the stitched panos that we have (>100 Megapixel). I would suggest not even thinking about a solution that does not permit this.

@Aklapper, this task ranked highly in the most recent community wishlist and the Community Tech team's subsequent prioritization process, so I'm not sure describing it as "lowest" is accurate. I certainly hope it's not considered that unimportant.

@Sdkb: Priority should reflect reality and does not cause it. If someone actively plans to work on this (it would be welcome), then they are welcome to increase priority.

My point is that the Community Tech team does plan to work on this, at least from what they've communicated. I've added it to their group and removed the priority so that they can assess it themselves.

In T138933#7377322, @Sdkb wrote:

My point is that the Community Tech team does plan to work on this, at least from what they've communicated. I've added it to their group and removed the priority so that they can assess it themselves.

We discussed this with the team, and no one was able to confirm we explicitly said we are going to take on this project. We apologize if anyone was misled. Yes, the wish performed better this year than in previous years, and we certainly we like to work on it (as with any popular wish), but we can't commit to that until current priorities are completed. In my opinion, unfortunately it seems unlikely we'd get to it before the next survey starts in January. If that turns out to be true, we ask that the wish be re-proposed, and after the survey we will prioritize it accordingly.

For future reference, we typically do not add the Community-Tech tag to wishlist tasks until we are sure we can commit to them. I have updated our project description accordingly. Thanks for your understanding,

Thanks for the update/clarification, @MusikAnimal. For reference, by "ranked highly in the most recent community wishlist and the Community Tech team's subsequent prioritization process" I was referring to this report. I certainly understand that the resources of the team are extremely limited, and I'm not saying this particular task should displace others, but it remains very frustrating that 91 different editors flagging a task as important is still not enough to get the WMF to allocate any resources at all to that task.

For reference, by "ranked highly in the most recent community wishlist and the Community Tech team's subsequent prioritization process" I was referring to this report.

Glad you saw that :) If it wasn't clear, that's not us committing to everything listed there, rather it's our own prioritized list from which we work from the top-bottom. Sorry for the confusion. If you have any further questions about this process or the wishlist in general, we'd love to engage, but we should discuss over on Meta to avoid more unrelated noise here. Thanks, and apologies again if we misled you or anyone else!

A point that recently came up on en-WP (kudos to Magnolia677) is that the version of the panorama viewer does not include an easily noticeable link to the Commons file page, only the enlarge button in the caption that's very rarely used. This makes it more difficult to see information (like description/date) of files and to access tools like the download button. A redesign of the rendering gadget should address this.

https://www.mediawiki.org/wiki/User:TheDJ/panoviewer
https://gitlab.wikimedia.org/hartman/panoviewer

I also found a new library worth exploring called panolens.js

In T138933#8390562, @TheDJ wrote:

https://gitlab.wikimedia.org/hartman/panoviewer

I also found a new library worth exploring called panolens.js

I know this may be beyond the scope of just this one task, but it would be nice to be able to support clicking on hotspots in the 360 panorama/photosphere to navigate to other ones (ie. virtual tour mode). Pannellum supports this quite nicely already. It would be pretty cool to see Commons community perhaps stitching multiple spheres for a location (Wiki Loves Monuments comes to mind) to provide a more immersive and interactive experience.

https://pannellum.org/documentation/examples/tour/

Some notes about tile generation:

Panellum's generate.py is pretty small, 350 lines. It takes the input file, which is most often some part of a sphere mapped to a plane with an equirectangular projection. It feeds the file to Hugin's nona with a script instructing it to reproject the image to cube faces, stored as 6 separate TIFF images. Then it uses Pillow to crop and rescale the cube faces to make tiles.

In T138933#2416893, @Yurik wrote:

Added a Mapnik issue, will see if anyone replies.

No useful replies. It's not going to happen. Reprojecting is pretty heavy and complicated. Best to stick with an upstream project that actually exists.

Cutting up some big images into tiles is pretty simple. I doubt it would make sense to share code with map tiling, since there would be a lot of differing assumptions.

I looked at the sizes of the files in the commons category, and the sizes of cache directories generated by the tool, with a view to deciding whether Shellbox is feasible or if we'd run into T292322.

The tool has a maximum source image width 3700px, above which it tries to get a downscaled image from thumb.php. With that kind of limit in production, it might be feasible to use Shellbox. But it doesn't seem ideal.

In T138933#5754444, @dschwen wrote:

Just as a quick comment, I think multires tiles are absolutely necessary for the stitched panos that we have (>100 Megapixel). I would suggest not even thinking about a solution that does not permit this.

I assume then that you would like to lift the 3700px width limit. With a typical aspect ratio of 2:1, 100 Mpx would give a size of 14142 x 7071.

The number of tiles per input file often exceeds 1000.

I looked at a small sample of 164 cache directories. The average was 431 tiles and the maximum was 2469 tiles.

Tile generation in the tool has been broken since April 2021, so the previous statistics about generated output only includes files generated up to that date.

I'm fixing it.

I submitted https://github.com/toollabs/panoviewer/pull/3

I don't think tile generation is necessary for initial production deployment. It's pretty complicated to do efficiently, and the user experience is reasonable without it, especially if you disable the zoom controls.

I mean, it would be nice to have, but we only have two more days allocated to this task as part of the CommTech wishathon.

Here's what I think is actually required:

• Detect spherical equirectangular images with UsePanoramaViewer=true. Ignore other images.
• Opening the image in MultimediaViewer should launch pannellum.
• There should be a parser image parameter which causes the thumbnail to show a projection with 100° horizontal field of view, matching the pannellum default. I think it would be a non-default option since showing the full image makes more sense for abuse control.
• Thumbnail overlay icon, so that the user knows there is more to see in the image.
• Pannellum localisation (see strings config)

So the following subtasks would be split off, to be done later, if they are still desired:

• Launching pannellum without MMV.
• Cylinders, stereoscopy.
• Exposing pannellum configuration parameters to the user.
• Additional metadata.
• Tile generation.

So I imagine most of it would either be an MMV patch or an extension which is very tightly integrated with MMV.

Thumbor will need to know how to generate a 100° FOV thumbnail.

• Patch MMV onThumbnailBeforeProduceHTML() to add an attribute when UsePanoramaViewer=true and ProjectionType=equirectangular. Use File::getMetadataItems().
• The MMV Canvas constructor doc comment says "it can be extended to other media types", so maybe try that. May need to split off a shared base class. Add a class that embeds Pannellum via its API. May need a factory instead of unconditional Canvas construction in LightboxInterface.
• MMV calculates thumbnail size and does an imageinfo request with iiurlwidth/iiurlheight. When the panoramic attribute is present we want to request a larger thumbnail. Maybe still related to the viewport size but scaled up by a suitable factor to account for content outside the default 100° viewport.
• Add a thumbnail parameter to core, maybe in ExifBitmapHandler or JpegHandler. Just need to accept it and add it to the URL, can ignore in core image scaling.

@tstarling if you were to integrate VIPS based large image tiling into the framework you could in one fell swoop support both tiled high res panoramics and the zoomviewer!

I've updated my Panoviewer extension a bit https://www.mediawiki.org/wiki/User:TheDJ/panoviewer

It's now self contained. Still needs a bit of cleanup before it can be deployed. And still doesn't have MMV support either.