I've added it to kanban so that you can pick it up when you are ready. Prio sounds good to me.

On IRC, @Pnorman suggested going higher than 3.6.x, to 3.7.x or 4.0.x, as long as we're going through the trouble of upgrading. That sounds good to me.

I've got a POC branch for 3.7.x at https://github.com/mdholloway/kartotherian/tree/deps-3.7 similar to the one above for 3.6.x. It seems to work well. We can't go straight to 4.0.x yet because our @mapbox dependencies haven't yet adopted it and are still on 3.7.x.

If going for 3.7.x is agreeable to everyone, I'll do the same for tilerator and do all the necessary bumping and publishing so we can get this pushed out to the new stretch maps beta tester (deployment-maps04). It'll also take a couple of patches in the deploy repos for scap, which I'll do shortly.

Tilerator POC branch for mapnik 3.7.x: https://github.com/mdholloway/tilerator/tree/deps-3.7

If going for 3.7.x is agreeable to everyone

@Mholloway I am okay with that.

In T188674#4428409, @Mholloway wrote:

Tilerator POC branch for mapnik 3.7.x: https://github.com/mdholloway/tilerator/tree/deps-3.7

I cloned your POC and tested, everything looks good to me. You can go ahead with the PR too.

Progress update:

The following repos have been updated, with version bumps as needed, but the updates are not yet published in npm:

• kartotherian/core
• kartotherian/maki
• kartotherian/snapshot
• kartotherian/tilelive-vector

The following will need updating/publishing when the dependency updates are published:

• kartotherian/kartotherian
• kartotherian/tilerator

To be continued tomorrow morning.

OK, so: I haven't been able to find the a way to successfully build mapnik from source when updating to node-mapnik 3.6 or 3.7, and on further reflection, I'm not sure this is actually quite the blocker to our other work that I thought it was:

• This PR will unblock T172090: move maps-test cluster to cloud infrastructure: https://github.com/kartotherian/kartotherian/pull/71
• Similarly, in the event the warnings in T195316: unmet peer dependency warning on kartotherian and tilerator builds become errors, most likely due to an npm upgrade in production, we can fork tilelive-tmstyle and have the offending peerDependencies instead point at our fork of tilelive-vector, and (non-namespaced) tilelive (if we haven't already moved to @mapbox/tilelive in the course of a mapnik upgrade by that time). Since it isn't causing any real problems now, and we know how to deal with it in the event it does, I think that one can have its priority lowered to "normal."

It would still be nice to upgrade mapnik, but it's worth noting that node-mapnik 3.5.14 is the version packaged for Stretch in the official Debian repos. 3.7.2 is packaged for Buster, which suggests that we might have an easier time building 3.7 against the system mapnik when we upgrade to Buster in production, whenever that happens.

What version of mapnik are you trying to build node-mapnik 3.7.2 against?

I'm looking at the instructions here:

https://github.com/mapnik/node-mapnik#source-build

IIUC, without any intervention from us, the mapnik version this is trying to build against when working with a stock debian:stretch base image will be this one, 3.0.12:

https://packages.debian.org/source/stretch/mapnik

I noticed in the README that "The master branch of node-mapnik is not compatible with 3.0.x series of Mapnik. To build again Mapnik 3.0.x, use v3.0.x branch," so I've tried specifying appropriate commits from the v3.0.x branch of node-mapnik (for 3.6.3, or the head of that branch) in addition to the published 3.6.2 and 3.7.2. Nothing so far has worked.

Looking further now, I see from node-mapnik's package.json @3.7.2 that it wants mapnik_version v3.0.20 or greater, so I guess it is no surprise that that's not working. 3.6.2 wants mapnik 3.0.15. (And 3.6.1 wants 3.0.14, and 3.6.0 wants 3.0.13.)

The thing about mapnik 3.0.x compatibility is confusing to me because it seems to contradict mapnik_version in package.json ever since it was added in 3dba76a. Maybe I'm missing something.

In any case it looks like we will need to find a backport of a later version of mapnik to get this working.

And looking at the Docker output, 3.0.12+ds-3 is indeed the version being pulled down:

Get:328 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 libmapnik3.0 amd64 3.0.12+ds-3 [2169 kB]
Get:356 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 libmapnik-dev amd64 3.0.12+ds-3 [674 kB]
Get:365 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 mapnik-doc all 3.0.12+ds-3 [2137 kB]
Get:366 http://cdn-fastly.deb.debian.org/debian stretch/main amd64 mapnik-utils amd64 3.0.12+ds-3 [228 kB]

So basically I've re-confirmed everything you (@Pnorman) wrote in the task description...

Did anything come of this?

I'm asking Debian GIS about backports for the system Mapnik.

In T188674#4438346, @Mholloway wrote:

The thing about mapnik 3.0.x compatibility is confusing to me because it seems to contradict mapnik_version in package.json ever since it was added in 3dba76a. Maybe I'm missing something.

Oh, maybe this is because mapnik_version has been referencing mapnik git commits rather than v3.0.x version references ever since this commit. Probably more of a node-mapnik dev concern.

Let's talk to @Gehel next week (if not sooner) about backporting mapnik 3.0.20 for Stretch.

Kartotherian and its dependencies are all upgraded and they were tricky to test because we can't have more than one version of node mapnik running in the same ecosystem.

• https://github.com/kartotherian/kartotherian/pull/73
• https://github.com/kartotherian/maki/pull/3
• https://github.com/kartotherian/snapshot/pull/13
• https://github.com/kartotherian/tilelive-tmsource/pull/1

I am also bumping the version of the libraries to v1.0.0 because of an upcoming task (T187478)

WIP: tilerator

@Jhernandez sure! I am on it.