[OSM] Setup a beta cluster instance to test imposm3 DB replication
Closed, ResolvedPublic3 Estimated Story Points
Actions

Assigned To

Authored By

	MSantos
	Aug 20 2020, 9:36 PM

Description

Background information

In order to move to imposm3, we need a beta cluster instance to test it.

What

Setup a beta cluster instance to test initial import and replication scripts with imposm3
Initially, this was a task to test imposm3 in the beta cluster, but these acceptance criteria were transferred to the actual tasks to introduce imposm3 into our puppet configuration.

Acceptance criteria

beta cluster is configured and available for tests

Conclusion

The beta cluster is setup at deployment-maps08.deployment-prep.eqiad1.wikimedia.cloud, to access the tile server in a public domain please refer to https://maps-beta.wmflabs.org/

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Add new target for beta environment and clean-up old envs	maps/kartotherian/deploy	master	+1 -6
	Add new target for beta environment and clean-up old envs	maps/tilerator/deploy	master	+1 -18

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved	BUG REPORT	None	T312751 Geoline never shows up after OSM and Wikidata entries have been updated
Resolved		None	T260456 [Maps] Reduce Map Sync Latency with OpenStreetMaps (OSM)
Resolved		MSantos	T260947 [OSM] Setup a beta cluster instance to test imposm3 DB replication
Resolved		hnowlan	T263726 [Bug] Puppet is failing for maps beta cluster - could not find group maps-admins

Event Timeline

MSantos created this task.Aug 20 2020, 9:36 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 20 2020, 9:36 PM

MSantos mentioned this in T238554: [Spike] Consider using imposm3 as the OSM replication system.Aug 20 2020, 9:56 PM

MSantos added a parent task: T260456: [Maps] Reduce Map Sync Latency with OpenStreetMaps (OSM).Aug 20 2020, 10:01 PM

• sdkim moved this task from Needs triage to Upcoming on the Product-Infrastructure-Team-Backlog-Deprecated board.Aug 26 2020, 3:37 PM

MSantos triaged this task as Medium priority.Aug 26 2020, 3:40 PM

I'm having some trouble to setup the beta cluster at deployment-maps07.deployment-prep.eqiad1.wikimedia.cloud

The puppet agent is failing with the following error:

Notice: /Stage[main]/Lvs::Realserver/Exec[/usr/sbin/dpkg-reconfigure -p critical -f noninteractive wikimedia-lvs-realserver]: Triggered 'refresh' from 1 event
Error: Could not find group maps-admins
Error: /Stage[main]/Profile::Maps::Postgresql_common/File[/var/log/postgresql/postgresql-9.6-main.log]/group: change from 'adm' to 'maps-admins' failed: Could not find group maps-admins

@Gehel and @RKemper any thoughts?

LGoto assigned this task to MSantos.Sep 23 2020, 3:55 PM

LGoto raised the priority of this task from Medium to High.

LGoto edited projects, added Product-Infrastructure-Team-Backlog-Deprecated (Kanban); removed Product-Infrastructure-Team-Backlog-Deprecated.

LGoto moved this task from To Do to Doing on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.

Blocked on T263726: [Bug] Puppet is failing for maps beta cluster - could not find group maps-admins

MSantos moved this task from All map-related tasks to Production Infrastructure on the Maps board.Sep 24 2020, 10:44 AM

MSantos mentioned this in T222377: Move kartotherian/tilerator logging to new logging pipeline.Sep 30 2020, 3:26 PM

MSantos mentioned this in T223041: Install a robots.txt file for maps.wikimedia.org.

MSantos mentioned this in T255932: Automatic zoom and positioning doesn't work for geomasks in thumbnails.

MSantos set the point value for this task to 3.Oct 6 2020, 11:06 AM

MSantos closed subtask T263726: [Bug] Puppet is failing for maps beta cluster - could not find group maps-admins as Resolved.Oct 13 2020, 1:46 PM

MSantos moved this task from Blocked to Doing on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.

MSantos updated the task description. (Show Details)Oct 14 2020, 9:59 AM

MSantos updated the task description. (Show Details)Oct 14 2020, 10:21 AM

MSantos moved this task from Doing to Code Review on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.

Change 633975 had a related patch set uploaded (by MSantos; owner: MSantos):
[maps/kartotherian/deploy@master] Add new target for beta environment and clean-up old envs

https://gerrit.wikimedia.org/r/633975

gerritbot added a project: Patch-For-Review.Oct 14 2020, 10:51 AM

Change 633976 had a related patch set uploaded (by MSantos; owner: MSantos):
[maps/tilerator/deploy@master] Add new target for beta environment and clean-up old envs

https://gerrit.wikimedia.org/r/633976

Change 633976 merged by jenkins-bot:
[maps/tilerator/deploy@master] Add new target for beta environment and clean-up old envs

https://gerrit.wikimedia.org/r/633976

Change 633975 merged by jenkins-bot:
[maps/kartotherian/deploy@master] Add new target for beta environment and clean-up old envs