Page MenuHomePhabricator

Figure out when to convert brighmed and meddo to XML
Closed, ResolvedPublic

Description

brighmed and meddo are written as a tm and tm2project respectively, and Kartotherian wants partial Mapnik XML. There's three options for when to convert it

  1. When loading in Kartotherian. Tilelive providers exist for tm2 and tm2source.
  1. At build time as part of package scripts
  1. At build time as part of NPM install scripts
  1. On commit time in the meddo and brighmed repos. This is against of the goal of making it possible for the community to contribute to the map style, as it adds a barrier with having to keep two files in sync.

My preference would be 1, or 3. 3 is easy, and I'm asking how easy 1 is.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

What tool does the conversion?

1 is easy to find out if those providers actually work. It's also the option with the best development experience for a casual user who uses the services to test his changes. (For example: change data.yaml, restart the service, see it in action. No extra step required.)

For both of them it's based on carto, but because neither is a complete Mapnik or CartoCSS style by themselves, it requires additional glue logic.

Normally the rendering software converts it internally, e.g. tessera reads them directly for serving tiles, and for design, Kosmtik reads the YAML as well. Kosmtik can produce XML, and it's what I use. It's not an ideal choice because it does a lot more than just that, so requires more.

The tm2 and tm2source formats have developed an ecosystem of tools that process them, some which don't even rely on node-mapnik or Mapnik

Is there a special URL providing the "brighmed" style?

The tilelive providers thinly wrap the common ones, converting to Mapnik XML, so I'm going to go ahead and test them out.

Is there a special URL providing the "brighmed" style?

brighmed is the name of the new cartography. There's a demo running at http://legolas.paulnorman.ca:6789/brighmed, but this issue is just about a specific part of what's needed to put it into production.

For options 2 and 3 (and maybe 4), there needs to be a command that we can run to do the conversion.

@Pnorman do you know what command can be used or is it part of this task to figure out? Also, could you share how exactly you've been doing it up to this point?

Here's the approach I propose to load tmsource yaml

config.yaml
modules:
  - tilelive-tmsource
sources.yaml
genraw:
  public: true
  formats: [pbf]
  uri: tmsource://
  yaml:
    npm: ["@kartotherian/osm-bright-source", "data.yml"]
  yamlSetDataSource:
    if:
      dbname: gis
      host: ''
      type: postgis
    set:
      host: localhost
      user: {var: osmdb-user}
      password: {var: osmdb-pswd}
  • Add YamlLoader to @kartotherian/module-loader
  • Make @kartotherian/core support src.yaml by using YamlLoader
  • Fork tilelive-tmsource and make it support receiving a yaml string instead of a yaml file path
  • Update tilerator to include tilelive-tmsource instead of tilelive-bridge. Update sample config and source files accordingly.
  • Update osm-bright-source yaml file to make it work (there is 1 small issue that tilelive-tmsource chokes on)

Here's the approach I propose to load tmstyle yaml

config.yaml
modules:
  - tilelive-tmstyle
sources.yaml
osm-intl:
  # omitted properties here
  uri: tmstyle://
  yaml:
    npm: ["@kartotherian/osm-bright-style", "project.yml"]
  yamlSetParams:
    source: {ref: osm-pbf}
  • Make YamlLoader support yamlSetParams, yamlLayers and yamlExceptLayers
  • Make tilelive-tmstyle accept yaml as string in uri object (instead of file path)
  • Update osm-bright-tm2 to make it ready to be loaded (inline fonts, etc)
  • Update kartotherian and tilerator to include tilelive-tmstyle as well as tilelive-vector (it still needs to be loaded to handle the vector:// uri to which tilelive-tmstyle delegates). Update sample config files.

This is now in production everywhere (labs, maps-test, maps both DC).