We have some kartotherian changes queued up, and would like to smoke test them somewhere public.
Implementation:
https://github.com/wikimedia/tilelive-tmstyle/pull/1
https://github.com/wikimedia/tilelive-vector/pull/3
https://gerrit.wikimedia.org/r/c/mediawiki/services/kartotherian/+/879271
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | None | T323920 Deploy recent map server work to production ( Jan 2023 ) | |||
| Open | None | T326290 Update error analysis after deploying robustness and logging improvements | |||
| Resolved | None | T323919 Deploy recent kartotherian to the beta cluster |
Change 868080 had a related patch set uploaded (by Awight; author: Awight):
[maps/kartotherian/deploy@master] Update kartotherian to cc25ba4
Change 868080 abandoned by Awight:
[maps/kartotherian/deploy@master] Update kartotherian to cc25ba4
Reason:
Needs updates
New work is on a branch of maps-kartotherian-deploy, called "2023-01-upgrades". Current status is that it's crashing with a node_modules / configuration error:
Cannot find module '@wikimedia/kartotherian-tilelive-switch'
Change 877186 had a related patch set uploaded (by Awight; author: Awight):
[maps/kartotherian/deploy@2023-01-upgrades] Remove deprecated modules
Change 877186 merged by jenkins-bot:
[maps/kartotherian/deploy@2023-01-upgrades] Remove deprecated modules
More issues:
Error: Cannot find module 'ajv'
....
/srv/deployment/kartotherian/deploy-cache/revs/db173177037b4569c4df7c7175fc461cf80d2b0d/node_modules/@kartotherian/tilelive-vector/node_modules/request/lib/har.js:5:16)
Change 878006 had a related patch set uploaded (by Awight; author: Awight):
[mediawiki/services/kartotherian@master] Reenable modules and example config to mirror traffic
Following up on the dependency issue, the beta deployment branch removes node_modules/ajv, but I'm not certain why. The service-runner-generated Dockerfile does npm install --production --build-from-source=@kartotherian/mapnik (skipping devDependencies), and "ajv" was added to devDependencies by the service-runner upgrade patch, but "--production" has been part of service-runner forever so I'm not sure why this change would have triggered a removal.
There's still a copy of "ajv" in node_modules because it's a dependency of request->har-validator, I'm not sure why it isn't seen by the npm module loader.
Change 878006 merged by jenkins-bot:
[mediawiki/services/kartotherian@master] Reenable modules and example config to mirror traffic
Happily, it's possible to see this dependency issue without a deployment, after building the deploy repo:
cd maps-kartotherian-deploy npm ls --omit=dev --all ... kartotherian@2.1.0 ├─┬ @kartotherian/tilelive-vector@4.0.3-alpha.0 │ ├─┬ request@2.83.0 │ │ ├─┬ har-validator@5.0.3 │ │ │ ├── UNMET DEPENDENCY ajv@^5.1.0
Here's a little hack to run the deployment repo using kartodock:
volumes: - /home/USER/kartotherian-deploy/src:/home/kartotherian - /home/USER/kartotherian-deploy/node_modules:/home/kartotherian/node_modules
The container must be rebuilt after the deployment is built, since the node_modules directory will be removed and recreated.
Using this arrangement, I can reproduce the error thrown on beta.
Interestingly, all errors seem to be related to tilelive-vector and the even older libraries--for example, har-validator is deprecated entirely. I'll try forking tilelive-vector and updating some of the dependencies.
42 error code ELSPROBLEMS 43 error missing: @mapbox/tilelive-vector@>=2.0.0, required by tilelive-tmstyle@0.8.0 43 error missing: ajv@^5.1.0, required by har-validator@5.0.3 44 verbose exit 1
Change 879271 had a related patch set uploaded (by Awight; author: Awight):
[mediawiki/services/kartotherian@master] [WIP] Switch to forked libs to allow patching
Change 879271 abandoned by Awight:
[mediawiki/services/kartotherian@master] Switch to forked libs to allow patching
Reason:
Was unnecessary.
Change 879560 had a related patch set uploaded (by Awight; author: Awight):
[mediawiki/services/kartotherian@master] [WIP] Play with "ajv" workaround
Change 879560 merged by jenkins-bot:
[mediawiki/services/kartotherian@master] "ajv" workaround
Change 879744 had a related patch set uploaded (by Awight; author: Awight):
[maps/kartotherian/deploy@2023-01-upgrades-2] Update kartotherian to 0ac5703
Change 879744 merged by jenkins-bot:
[maps/kartotherian/deploy@2023-01-upgrades-2] Update kartotherian to 0ac5703
Change 879750 had a related patch set uploaded (by Awight; author: Awight):
[maps/kartotherian/deploy@2023-01-upgrades-2] Remove deprecated modules
Change 879750 merged by Awight:
[maps/kartotherian/deploy@2023-01-upgrades-2] Remove deprecated modules
Change 879753 had a related patch set uploaded (by Awight; author: Awight):
[maps/kartotherian/deploy@2023-01-upgrades-2] Restore tilelive-switch
Change 879753 merged by jenkins-bot:
[maps/kartotherian/deploy@2023-01-upgrades-2] Restore tilelive-switch
Change 879760 had a related patch set uploaded (by Awight; author: Awight):
[maps/kartotherian/deploy@2023-01-upgrades-2] Turn mirroring off and supply missing variable
Change 879760 merged by jenkins-bot:
[maps/kartotherian/deploy@2023-01-upgrades-2] Turn mirroring off and supply missing variable
Now on the Beta Cluster.
Example new logline,
[2023-01-13T11:07:32.446Z] WARN: kartotherian/122 on deployment-maps-master01: Received failed group: HTTPError: Unexpected token S in JSON at position 7921664 with URL https://maps-beta.wmflabs.org/geopoint?getgeojson=1&query=SELECT+distinct+%3Fid+%3Fgeo+WHERE+%7B%3Fid+wdt%3AP31%2Fwdt%3AP279%2A+wd%3AQ16970%3B+wdt%3AP625+%3Fgeo.+%3Fid+p%3AP131+%3Fstatement1.+%3Fstatement1+%28ps%3AP131%2F%28wdt%3AP131%2A%29%29+wd%3AQ38.%7D (levelPath=warn, request_id=51f9b2c0-9332-11ed-98fa-11e97f051bc4)
Change 884852 had a related patch set uploaded (by Awight; author: Awight):
[maps/kartotherian/deploy@master] Remove deprecated modules
Change 884852 merged by jenkins-bot:
[maps/kartotherian/deploy@master] Remove deprecated modules