Page MenuHomePhabricator

[SPIKE] Investigate LOE and Risks to reintroduce Nearby
Closed, ResolvedPublic

Assigned To
Authored By
JTannerWMF
Nov 30 2022, 12:56 AM
Referenced Files
F36703290: Screenshot_20230203_101755.png
Feb 3 2023, 4:27 PM
F36563116: Screenshot_20230130_203627.png
Jan 31 2023, 1:42 AM
F36201891: image.png
Jan 13 2023, 9:26 PM
F36201875: image.png
Jan 13 2023, 9:26 PM
F36201274: Screenshot_20230113_135147.png
Jan 13 2023, 7:16 PM
F36201272: Screenshot_20230113_135107.png
Jan 13 2023, 7:16 PM
F36196792: image.png
Jan 13 2023, 7:16 PM

Description

Background

Android had Nearby/Places in the app several years ago. It was removed due to performance challenges (details can be found here). It is a possibility that open source maps have improved since the feature has been removed OR the tolerance towards Google Maps may have change (more context can be found here).

Task

  • Reach out to @kchapman for guidance on open source maps to use
  • Investigate using Openstreetmaps and putting restrictions on scroll
  • Touch base with Legal about using Google Maps and Opt-Out

Event Timeline

LGoto renamed this task from [SPIKE] Investiage LOE and Risks to reintroduce Nearby to [SPIKE] Investigate LOE and Risks to reintroduce Nearby.Dec 5 2022, 5:17 PM
JTannerWMF added a subscriber: Dbrant.

Reassigning to @Dbrant so he can look into the updates with open street maps and if restricting scroll has an impact on performance

A few notes so far:

Google Maps

➕ Very easy to integrate.
➕ Minimal maintenance, and modest increase in total APK size.
➖ Won't work on devices without Google services.
➖ Not open-source, so won't be allowed in F-Droid builds.
➖ Not free to use. Here is Google's pricing structure for Maps.

Mapbox

Mapbox is the best-known SDK (outside of Google) for integrating maps into web and native platforms, and this is what we were using previously to hook into our own OpenStreetMap tileserver.
(To clarify a bit: Mapbox is a company that offers components and SDKs for rendering maps on web and native platforms, and they also have their own tile servers, but you can also just take their SDK and point it to your own tile server, which is what we were doing.)

However, there have been some recent changes to Mapbox's licensing model:

➖ We can no longer use the Mapbox library without creating an account with them and using an API key (even if we use our own tile servers).
➖ When I attempted to create/access my account (just for the purpose of trying their latest library) I was greeted with this:

image.png (953×706 px, 96 KB)

➖ According to recent discussions, it sounds like Mapbox now charges fees for every map-load, even if using your own tile server.

For these reasons we can conclude that Mapbox might no longer be the right fit for our needs.

MapLibre

There is now a community-maintained fork of the Mapbox SDK that will remain open-source, but probably won't receive backports from upstream. Nevertheless this is probably closer to what we're looking for. The rough pros and cons of using this SDK will be:

➕ Totally free and usable in all builds, including on devices that don't have Google services.
➕ Relatively easy to resurrect based on our old code, since integration will be very similar.
➖ By necessity, less functional than the standard Google Maps app on the device. (e.g. "Can I get directions to this marker from my current location?" "Can I have a satellite view instead of the regular map?" "Is there a traffic overlay?" etc)
➖ Nearly doubles APK size from the current 14 MB to 25 MB.
➖ Will require more maintenance; Any errors or issues with tiles will be ours to fix.

Here's a quick example of it in action, using raster tiles from the Wikimedia tile server:

Screenshot_20230113_135107.png (1×720 px, 1 MB)
Screenshot_20230113_135147.png (1×720 px, 1 MB)

As you can see, using raster tiles is suboptimal and comes with quirks: Rotating the map away from true North doesn't keep text labels aligned horizontally; Some tiles have cut-off labels from one tile to the next, etc.

@kchapman Do our tile servers produce vector tiles today? Is there a way to specify use of vector tiles in a TileJSON configuration? Apologies, I'm out of date on the state of Maps and their configuration. Here was our old configuration for getting raster tiles:

{
    "type": "raster",
    "tiles": [
        "https://maps.wikimedia.org/osm-intl/{z}/{x}/{y}.png"
    ],
    "tileSize": 128
}

; Any errors or issues with tiles will be ours to fix.

Presumably you mean (rare) errors in tile rendering, rather than (far more common) mapping errors, which will fall to the OSM community?

Otherwise, please clarify.

Presumably you mean (rare) errors in tile rendering, rather than (far more common) mapping errors, which will fall to the OSM community?

Yes, sorry, I do mean errors in tile rendering, but I wouldn't call them rare; they seem to occur everywhere, particularly at certain zoom levels like 11 and 12.

In this map of London and its surroundings, I can count at least eight labels that are cut off incorrectly:

image.png (845×1 px, 1 MB)

And here in Chicago, a couple of cut-off labels, and the whole majority of Chicago weirdly lacking any labels:

image.png (864×1 px, 1 MB)

@Dbrant we do have support for vector tiles [1] and an effort on the MaplibrGL styling to match current Wikipedia Maps (T276317).

That might be sufficient for a prototype using vector tile rendering on the client-side. But please let me know if I can provide more support.

[1] Endpoint example for vector-tile rendering https://maps.wikimedia.org/osm-intl/4/7/8.pbf

EDIT: link to the maplibre style json https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Kartographer/+/678842

[1] Endpoint example for vector-tile rendering https://maps.wikimedia.org/osm-intl/4/7/8.pbf

This URL gives me the message "Format pbf is not known". Can you confirm?

[1] Endpoint example for vector-tile rendering https://maps.wikimedia.org/osm-intl/4/7/8.pbf

This URL gives me the message "Format pbf is not known". Can you confirm?

My bad, it should be https://maps.wikimedia.org/osm-pbf/4/7/8.pbf.

Can you check again?

@MSantos ooh very nice, thanks. I've got basic functionality up and running with WMF vector tiles 👍

Screenshot_20230130_203627.png (2×1 px, 1 MB)

OK, considering that we should now be able to use vector tiles in our map, let's restate the pros and cons of the maplibre approach:

Pros
  • Since we have vector tiles, zooming and rotating is now fluid and responsive. Also, since these are vector tiles, we can apply arbitrary styling to them, including things like "dark mode" and other themes in our maps, or highlighting particular overlays with different styles. Here's me twiddling the knobs on some random colors:

Screenshot_20230203_101755.png (2×1 px, 1 MB)

  • Integrating will be relatively easy. It's a simple matter of resurrecting our old code and bringing it up to date with current best practices. The interface with the MapLibre SDK hasn't changed significantly from when we were using Mapbox earlier.
Cons
  • Will increase APK size to 25 MB. Note, however, that this will still keep us on the low, low end of the app bloat spectrum.
  • Will be missing most of the functionality that's expected in a "maps" app. For any interaction other than "view the Wiki article for this marker", we would need to bounce the user out to their default external Maps app.

Also, one more question for @MSantos: There seem to be a couple of bits of the styling configuration that are dependent on a third party, namely:

"sprite": "https://petersongis.github.io/Mobility/wikisprites",
"glyphs": "https://petersongis.github.io/Mobility/fonts/{fontstack}/{range}.pbf",

Is this / will this be available on a WMF domain?

Just making a note for myself that I need to get the timeline for bringing fonts and sprites in house from another team

JTannerWMF added a subscriber: Seddon.

I'm actually going to move this to Needs Engineering Manager for @Seddon to let us know what the ETA is for Sprites and Fonts