Page MenuHomePhabricator

Kartotherian should accept (and propagate) a language parameter for tiles
Closed, ResolvedPublic

Description

We need to add an extra parameter that indicates which language the labels should be rendered in, and propagate that so that T187595: Implement a language selection processing step for vector tiles can use it.

The current URL scheme is https://maps.wikimedia.org/osm-intl/18/41940/101311.png . We could either add the language as a query string parameter like https://maps.wikimedia.org/osm-intl/18/41940/101311.png?lang=es , or we could add it as a path component like https://maps.wikimedia.org/osm-intl/es/18/41940/101311.png . I prefer the latter, but we will have to more explicitly provide backwards compatibility to keep old-style URLs working.

Event Timeline

Catrope triaged this task as Medium priority.Feb 16 2018, 11:23 PM
Catrope created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 16 2018, 11:23 PM
jmatazzoni raised the priority of this task from Medium to High.Feb 26 2018, 11:55 PM
SBisson closed this task as Invalid.Mar 14 2018, 12:58 PM
SBisson added a subscriber: SBisson.
SBisson reopened this task as Open.Mar 15 2018, 2:23 PM

Not so fast... while kartotherian/server does opts.lang = req.query.lang;, it's only passing opts to the target source, which has the protocol vector:// (from tilelive-vector). Sources don't propagate options by default to upstream sources. So the lang parameter stops right there and never reaches the babel source.

Yurik added a subscriber: Yurik.Mar 15 2018, 6:25 PM

@SBisson the custom version of vector should handle that, but I don't remember if that was fully implemented (most of it was for sure). See https://github.com/kartotherian/tilelive-vector ...

Just a thought - tilelive-vector has a lot of functionality that Kartotherian is not using, but at the same time has (or strives to have) better param passing. I think it might make sense to fork it completely... Not fully sure about this one yet.

@Yurik I really don't know enough to support or oppose using that fork...

My initial quick & dirty way to get it to work is to introduce a type of param called "query" in core/lib/sources.js, similar to npm, var, env, ref, loader, etc.

It is configure like:

genviewraw:
  uri: babel://
  params:
    source: {ref: gen}
    lang: {query: lang}

Then, when a request is being served, server/lib/tiles.js resolves those parameters from req.query and assigns the values to the appropriate source handlers. The main difference is that the source reads it from this.lang instead of opts.lang.

Yurik added a comment.Mar 15 2018, 7:42 PM

Interesting idea! The problem is that this (IIRC) is not per request, it's per tilelive instance. Each instance gets its state from the config file, so you have as many instances as sources. Per-request state is only passed via parameters, and that's where the limitation comes in - original tilelive API params are only x,y,z, not opts. That's why I had to introduce a new getAsync method to pass additional state.

Damn. That's right, this.lang would get reassigned every time a new request comes in and babel would read whatever value is the latest, instead of the one associated with the request it's currently serving.

Yurik added a comment.Mar 15 2018, 8:56 PM

Not exactly - this.lang always depends on which "source" you are talking about. The this.lang in babel instance is different from the this.lang in tilelive-vector instance. The pipeline is initiated during the startup - the global storage stores one instance per "source ID", and if one source gets data from another source, that chaining is set up initially. E.g. if view is using generator, there will be one instance of view, which internally references one instance of generator. When user asks for a tile, Kartotherian gets the named instance (e.g. "view" or "osm-intl"), and calls getTile (now its getAsync), passing all the params. Kartotherian would not be able to set any params on the "generator" source because at that point it doesn't know that generator source will be used by the view source.

When I wrote this.lang I was thinking in terms on the Babel instance that handles the source with the babel:// protocol ('genviewraw' in my case) but I realize it was not really clear. I think we are saying the same thing. Attaching request state to the source handler instance can work in the simplest case but it's not safe when serving concurrent requests.


I just tried using @kartotherian/tilelive-vector and it does propagate the opts to upstream sources. I now consider this a serious option for the following reasons:

  1. It propagates opts (this ticket)
  2. It includes a critical dependency update (T176168)
  3. The hope of staying up to date with the upstream component is null for now since they have upgraded to versions of mapbox/node-mapbox that we cannot use. Migrating to use it would apparently be a small project in itself.

@SBisson - is the Console errors described below something not relevant or can be just ignored?
The example from the task description - https://maps.wikimedia.org/osm-intl/18/41940/101311.png?lang=es (and the same link with ?lang=es)- gives numerous Console errors like the follows:

101311.png:1 Refused to apply inline style because it violates the following Content Security Policy directive: "style-src 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-7aM9H8iBwRpsamP3ufblK/cy1/f/10TFBKnybYmxB9Y='), or a nonce ('nonce-...') is required to enable inline execution.

Another example - https://maps.wikimedia.org/osm-intl/es/18/41940/101311.png - will display "Unknown source" along with the above errors in the Console.

@SBisson - is the Console errors described below something not relevant or can be just ignored?
The example from the task description - https://maps.wikimedia.org/osm-intl/18/41940/101311.png?lang=es (and the same link with ?lang=es)- gives numerous Console errors like the follows:

101311.png:1 Refused to apply inline style because it violates the following Content Security Policy directive: "style-src 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-7aM9H8iBwRpsamP3ufblK/cy1/f/10TFBKnybYmxB9Y='), or a nonce ('nonce-...') is required to enable inline execution.

I have no idea what those errors are about. This URL returns only an image. I don't know which inline style they're talking about.

Another example - https://maps.wikimedia.org/osm-intl/es/18/41940/101311.png - will display "Unknown source" along with the above errors in the Console.

This URL is invalid.

Etonkovidova closed this task as Resolved.Mar 27 2018, 4:40 PM