Page MenuHomePhabricator

Packaging PostGIS 3.1 for the new Maps stack
Open, MediumPublic

Description

The Maps team would ideally like to upgrade PostGIS to v3.1 since I believe it comes with performance improvements. This is part of the Maps stack upgrade work some of which is already underway and parts of which have been completed. We prefer that the new stack run on buster, not stretch, so we can start off without needing to worry about backports, etc.

The reason behind the PostGIS version is some performance improvement we would like to use in our spatial queries ( asMVT queries and feature IDs ).
According to the PostGIS docs, PostGIS 3.1 is compatible with Postgres 11 which is already available for debian buster.

Filing this task for the service-ops team to explore the effort involved in packaging this. On IRC, @MoritzMuehlenhoff said that PostGIS 3.1 is already packaged in debian official.

Event Timeline

Pasting in my comments from IRC (which are based on a very quick look):
[16:33] <moritzm> bullseye has 3.1, but it's hard to tell how complex a backport will be, given that maps is still on stretch
[16:33] <moritzm> and the list of build deps in 3.1 is not small
[16:33] <moritzm> and includes things like protobuf and libgdal
[16:34] <moritzm> I think this can only really be estimated if someone gives it a shot for half an hour

@MoritzMuehlenhoff you can assume that maps will be in buster by the time we need PostGIS v3.1. @hnowlan has already started the needed work (T269582: [OSM] perform imposm3 migration in production) and we are very close to having a safe environment to keep going with the Buster migration.

Change 676395 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Add postgresql-server-dev-all to package builder packages

https://gerrit.wikimedia.org/r/676395

@MSantos You can find a backport for buster at https://people.wikimedia.org/~jmm/postgis/ Can you run some tests whether that's what you need? If all is fine, I'll import them to a repository component on apt.wikimedia.org

Change 676395 merged by Muehlenhoff:

[operations/puppet@production] Add postgresql-server-dev-all to package builder packages

https://gerrit.wikimedia.org/r/676395

@MSantos You can find a backport for buster at https://people.wikimedia.org/~jmm/postgis/ Can you run some tests whether that's what you need? If all is fine, I'll import them to a repository component on apt.wikimedia.org

@MoritzMuehlenhoff just installed in the beta cluster and performed some tests and it works fine in a Debian Buster machine running PostgreSQL 11.

gis=# SELECT version();
                                                     version                                                      
------------------------------------------------------------------------------------------------------------------
 PostgreSQL 11.11 (Debian 11.11-0+deb10u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit

gis=# SELECT Postgis_full_version();
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="3.1.1 aaf4c79" [EXTENSION] PGSQL="110" GEOS="3.7.1-CAPI-1.11.1 27a5e771" PROJ="Rel. 5.2.0, September 15th, 2018" GDAL="GDAL 2.4.0, released 2018/12/14" LIBXML="2.9.4" LIBJSON="0.12.1" LIBPROTOBUF="1.3.1" WAGYU="0.5.0 (Internal)" RASTER
(1 row)

Change 677808 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] New component for PostGIS 3 backport

https://gerrit.wikimedia.org/r/677808

Change 677808 merged by Muehlenhoff:

[operations/puppet@production] New component for PostGIS 3 backport

https://gerrit.wikimedia.org/r/677808

Mentioned in SAL (#wikimedia-operations) [2021-04-08T08:17:51Z] <moritzm> imported postgis 3.1.1+dfsg-1~wmf1 to component/postgis for buster-wikimedia T277064

@MSantos, @hnowlan : I've uploaded the postgis 3.1.1 backport to the newly created component/postgis for buster. You can add it to the maps Puppet manifests using apt::package_from_component (feel free to add me as reviewer).

fgiunchedi triaged this task as Medium priority.Tue, Apr 13, 1:53 PM