Page MenuHomePhabricator

Please install Postgresql on Tool-labs
Closed, ResolvedPublic

Description

A installation of Postgresql with PostGIS and hstore would be nice to work with Wikipedia-coordinates there.

I assume the amount of data will be relatively small compared with the data we will have later from OpenStreetMap on an other Server.


Version: unspecified
Severity: normal

Details

Reference
bz48896

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 1:37 AM
bzimport added a project: Toolforge.
bzimport set Reference to bz48896.
Kolossos created this task.May 28 2013, 2:50 PM
coren added a comment.Aug 29 2013, 8:10 PM
  • Bug 51885 has been marked as a duplicate of this bug. ***
coren added a comment.Aug 29 2013, 8:10 PM

We are currently working on setting up a Mysql DB on physical servers for Labs'
use; given that there are also nice things that can be done with a working
pgsql in general (testing mw against it comes to mind), I think this should be
provided as a general service.

The current mysql DB plan calls for a master/slave setup, it'd be easy to set
up a postgres server on those same servers as well -- switching roles so that
performance is distributed.

Nice things possible with psql would be:
*Have PostGIS to play with Wikipedia-coordinates or other geospatial data as mention before.
*pgsql has a nice key-values store (hstore)
*pgsql starts to support JSON[1]. This is perhaps a really nice way to make data from WikiDATA accessible.

[1]http://www.postgresql.org/docs/9.3/static/functions-json.html

[moving from "Tools" to "WM Labs > tools"]

sumanah wrote:

Silke, what tools are waiting on this?

Silke: Do you know tools which are waiting for this (Postgres on Tool labs)?

silke added a comment.Nov 8 2013, 5:20 PM

This is for the part of OSM that is going to move to the Wikimedia Labs.

One tool which is waiting for it is Wikipedia coordinates [1], which is with 6 Mio. entries relatively small but often used. This tool is independent from OSM and was the reason for me to open this ticket, because it seems to be an easy task.

Other tools like WIWOSM, Addtags, ... need off course the full OSM database.
Existing OSM tools can be found here [2] (by using the search function).

If the database in WMF production is now running I hope we can get OSM database in Labs really soon.

Greetings

[1]https://de.wikipedia.org/wiki/Wikipedia:WikiProjekt_Georeferenzierung/Wikipedia-World/en

[2]https://www.mediawiki.org/wiki/Wikimedia_Labs/Tool_Labs/List_of_Toolserver_Tools

Is there an schedule for it? I read something about Postgresql-server for Labs in the WMF engineering report of November.

coren added a comment.Dec 16 2013, 7:03 PM

The hardware has been set aside for it in eqiad (the datacenter labs is moving to). Since we are moving all of labs, it didn't seem reasonable to set it up for a month and then have to migrate it, so the postgresql instance will be available in the new location (we are gunning for early Feb).

Thanks for the info.

silke added a comment.Feb 5 2014, 9:05 AM

Postgresql might be also needed to keep the toolserver wiki somewhere in Tool Labs (see #60220). Giving high priority for after the migration to equiad.

PostgreSQL master/server databases are good and ready to go
labsdb1004 is the master, labsdb1005 is the slave. Streaming asynchronous replication is setup between them, as configured by the puppet module.

Some minor manual steps to get the architecture fully working will be documented in the puppet module. PostGIS is scheduled to be enabled soon. OSM syncing from planet.osm still needs some work and will be ready ASAP.

silke added a comment.May 8 2014, 2:56 PM

So, can this ticket be closed?

Indeed it can. Closing it. Thanks Silke