Page MenuHomePhabricator

Implement a language selection processing step for vector tiles
Closed, InvalidPublic

Description

After fetching a vector tile from Cassandra and before rendering it, Kartotherian needs to process the tile to show labels in the correct language. This means modifying each feature that has a name_ field (containing a JSON blob of labels in all languages), and overwriting its name field with the right language. How it knows what the right language to select is is a combination of a URL parameter that needs to be propagated and a fallback chain, but those are separate tasks.

Event Timeline

Catrope triaged this task as Lowest priority.Feb 16 2018, 11:16 PM
Catrope created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 16 2018, 11:16 PM
Yurik added a subscriber: Yurik.Feb 16 2018, 11:53 PM

Kartotherian's babel does exactly this, but it needs to be configured with the right fallback rules.

jmatazzoni raised the priority of this task from Lowest to High.Feb 26 2018, 11:56 PM

@Yurik Could you advise on how exactly to configure Babel's json2tags:// and babel:// in the production sources config file? Thanks!

Yurik added a comment.Mar 14 2018, 2:42 PM

@SBisson I wrote some docs in https://github.com/kartotherian/babel#usage-examples -- I think prod source configs should follow that pattern. The tilerator config should use the json2tags as part of the tile generation pipeline - converting one giant json string into proper pbf tags. The babel should be used in kartotherian config, to convert the tile on the fly for the user. I would recommend using a language map option with it - to create a proper language fallback map.

@Yurik that makes sense. I guess I was hoping to take a shortcut and make it work with kartotherian only as a first step. This is how I am trying to hook things up:

# Generates vtiles from database
genraw:
  public: true
  formats: [pbf]
  uri: bridge://
  xml:
    npm: ["@kartotherian/osm-bright-source", "data.xml"]

# expand name_ attributes into individual tags
gen:
  uri: json2tags://
  params:
    source: {ref: genraw}
    tag: name

# set name tag based on name_* attributes
genviewraw:
  uri: babel://
  params:
    source: {ref: gen}
    tag: name
    languageMap:
      fr: ['es', 'en']

# View tiles as generated directly from the database. Don't view zoom0-5
genview:
  public: true
  formats: [png,json,headers,svg,jpeg]
  scales: [1.3, 1.5, 2, 2.6, 3]
  static: true
  maxheight: 2048
  maxwidth: 2048
  uri: vector://
  xml:
    loader: "@kartotherian/osm-bright-style"
  xmlSetParams:
    source: {ref: genviewraw}

Does make sense conceptually? Do you see anything obviously wrong?

I get a tile back but it never contains any label. I'm not sure where to look to start troubleshooting...

Yurik added a comment.Mar 14 2018, 3:38 PM

That seems correct. Try looking at the tile at every state, see if labels are there initially, and if so, when does it disappears

SBisson closed this task as Invalid.Mar 22 2018, 12:17 PM

This was already working in Babel