Maniphest T321959

Tech Wishes - Maps service infrastructure deprecations
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	• SWakiyama
	Oct 28 2022, 10:49 PM

Description

Request Status: New Request
Request Type: Support Kartographer/Kartotherian (maps)
Related OKRs: WMDE Better support for geo-information

Request Title: Tech Wishes Blocker

Request Description:

Error/bug reports related to geoshapes/points not appearing on maps consistently. After some investigation, WMDE Tech Wishes Team suspects several of the issues users are currently facing related to the reliability of maps on Wikimedia projects might be fixed by ensuring stable postgres replication, and refresh the three replicas which have stale date (last successful replication was March 2022).

A number of important upgrades are needed to keep maps running, including upgrading maps servers to supported Debian stable (bullseye), upgrading to a supported version of Node (v14), upgrading libmapnik to version 3.1.

Indicate Priority Level: High
Main Requestors: WMDE Technical Wishes / @Lena_WMDE
Ideal Delivery Date: November - December
Stakeholders: Maps users

Request Documentation

Document Type	Required?	Document/Link
Related PHAB Tickets	Yes	T316365: Restore the map data health and parity between clusters T321789: Upgrade maps servers to node >= 14 T216826: Move Kartotherian to Kubernetes
Product One Pager	Yes	https://meta.wikimedia.org/wiki/WMDE_Technical_Wishes/Geoinformation
Product Requirements Document (PRD)	No	<add link here>
Product Roadmap	No	<add link here>
Product Planning/Business Case	No	<add link here>
Product Brief	No	<add link here>
Other Links	No	<add links here>

Related Objects
Search...

Status	Assigned	Task
Open	None	T321959 Tech Wishes - Maps service infrastructure deprecations
Resolved	None	T316365 Restore the map data health and parity between clusters
Resolved	jijiki	T314472 Re-import full planet data into eqiad and codfw
Resolved	Jgiannelos	T324459 Enable traffic mirroring from codfw to eqiad
Resolved	jijiki	T325309 High error rate on maps.wikimedia.org
Resolved	awight	T321722 Look into the replica sync fails
Resolved	jijiki	T290149 Configure replication slots on Postgres masters
Resolved	jijiki	T321885 Resync stale maps postgres replicas
Open	None	T321887 Export maps postgres logs to logstash
Resolved	awight	T321888 Alert when maps postgres replicas are lagging
Open	None	T321789 Upgrade maps servers to node >= 14
Duplicate	None	T210815 Upgrade kartotherian to latest upstream node-mapnik
Open	None	T213077 Migrate Kartotherian to node-mapnik >v4.2.1 to support libmapnik >3.1.x
Resolved	MSantos	T209990 Test Kartotherian/Tilerator with node 10
Invalid	None	T195316 unmet peer dependency warning on kartotherian and tilerator builds
Resolved	MSantos	T188674 Upgrade node-mapnik to 3.7.x
Resolved	• Mholloway	T200284 Install mapnik 3.0.20 on Stretch
Resolved	None	T187481 Figure out Maki's Mapnik requirements
Resolved	MSantos	T215523 Upgrade node to version 10 on Beta Cluster for maps
Resolved	MSantos	T215852 Migrate Kartotherian/Tilerator to Node 10
Invalid	None	T216521 Create Debian packages for Node.js 8 upgrade for Maps
Resolved	awight	T322207 Get npm tests running for upgraded kartotherian
Resolved	None	T322208 Smoke test all different routes and endpoints of the mapnik-3.1 server
Resolved	WMDE-Fisch	T322398 Fix /geoshape /geoline /geopoint endpoints on mapnik-3.1
Resolved	lilients_WMDE	T322399 Create Wikipages to quickly smoketest routes and endpoints
Resolved	WMDE-Fisch	T322401 Fix tile routes that server non png output on mapnik-3.1
Open	None	T322615 Put upgraded mapnik 3.1 kartotherian on the beta cluster
Open	Jgiannelos	T327396 Migrate Kartotherian to node-mapnik v4.2.1 and unfork
Open	None	T216826 Move Kartotherian to Kubernetes
Resolved	• Mathew.onipe	T223275 Create blubberfile for deploying kartotherian into docker environment.
Invalid	None	T231009 Make jobprocessor's test not depend on external files
Open	None	T322522 Deploy containerized Kartotherian service to the beta cluster
Duplicate	None	T231006 Create helm chart for kartotherian k8s deployment
Resolved	None	T324471 Migrate our draft charts to newer scaffolding
Open	None	T322647 Create a dedicated postgresql+postgis cluster for maps
Open	None	T323360 Provision a hot-reload local development environment in k8s for kartotherian
Open	None	T370395 Investigation: updated usage overview for Kartotherian maps
Resolved	elukey	T378944 Strategy to slowly move Kartotherian's traffic from bare metal to k8s
Resolved	elukey	T381257 Allow Kartotherian to use a local HTTP proxy
Resolved	awight	T323981 Document outstanding maintenance for the maps stack

Event Timeline

• SWakiyama created this task.Oct 28 2022, 10:49 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 28 2022, 10:49 PM

• SWakiyama renamed this task from <SAMPLE TICKET> add request short name to Tech Wishes - Maps Errors/Bugs.Oct 28 2022, 10:50 PM

@SWakiyama, @DAbad: Could you please, in general, semantically connect related Phab tickets via Edit Related Tasks..., so related tasks (in this case T316365) don't remain completely disconnected? This also allows having a dependency graph and get notifications if related tasks get resolved. Thanks a lot.

AOdit_WMF added a project: WMDE Requirements on WMF needing prioritization.Oct 31 2022, 2:58 PM

dr0ptp4kt subscribed.Oct 31 2022, 5:57 PM

Lena_WMDE added a subtask: T316365: Restore the map data health and parity between clusters.Nov 1 2022, 1:22 PM

Lena_WMDE subscribed.

awight updated the task description. (Show Details)Nov 2 2022, 1:10 PM

elal subscribed.Nov 7 2022, 8:44 AM

awight updated the task description. (Show Details)Nov 8 2022, 3:19 PM

awight added subtasks: T321789: Upgrade maps servers to node >= 14, T216826: Move Kartotherian to Kubernetes.Nov 8 2022, 3:21 PM

awight renamed this task from Tech Wishes - Maps Errors/Bugs to Tech Wishes - Maps service infrastructure deprecations.Nov 8 2022, 3:24 PM

awight updated the task description. (Show Details)

awight updated the task description. (Show Details)Nov 8 2022, 3:27 PM

awight added a subtask: T323981: Document outstanding maintenance for the maps stack.Nov 29 2022, 9:49 AM

awight closed subtask T323981: Document outstanding maintenance for the maps stack as Resolved.Jan 5 2023, 8:39 AM

awight reopened subtask T323981: Document outstanding maintenance for the maps stack as Open.Jan 5 2023, 8:52 AM

awight closed subtask T323981: Document outstanding maintenance for the maps stack as Resolved.Jan 6 2023, 12:50 PM

Lena_WMDE added a project: WMDE-GeoInfo-FocusArea.Jan 18 2023, 9:09 AM

Lena_WMDE moved this task from Backlog to Needs WMF on the WMDE-GeoInfo-FocusArea board.

WMDE-Fisch closed subtask T316365: Restore the map data health and parity between clusters as Resolved.Mar 20 2023, 11:37 AM

dr0ptp4kt unsubscribed.Jul 28 2023, 4:08 PM

thiemowmde added a project: Maps.Feb 16 2024, 1:06 PM

Lina_Farid_WMDE subscribed.Apr 8 2024, 12:32 PM

WMDE-GeoInfo-FocusArea is archived and no activity on this ticket for 18 months - I guess this is dead and can be closed?

(...and are Foundational Technology Requests regularly groomed, dear WMF?)

In T321959#9993699, @Aklapper wrote:

WMDE-GeoInfo-FocusArea is archived and no activity on this ticket for 18 months - I guess this is dead and can be closed?

There's some recent activity in T216826 so I'd keep this open.

Tech Wishes - Maps service infrastructure deprecationsOpen, Needs TriagePublicActions