Page MenuHomePhabricator

Build a Docker container providing proxy access to Wiki Replicas
Closed, ResolvedPublic

Description

The Wiki Replicas are replicas of the production databases. Many tools on Toolforge use them.

Tool developers may use Docker and docker-compose for local development environment (see eg rTHER or R1969). But once the tool attempts to connect to the replicas, the connection fails.

The documentation already suggests to use SSH tunneling for local testing which makes use of Wiki Replica databases

Idea: providing a Docker image that does the proxying, which you can attach to your compose project and get replicas access :)

Event Timeline

FYI @dmaza created something like this here:
https://github.com/wikimedia/InteractionTimeline/blob/master/Dockerfile-replica-db
but right now it embeds your SSH key and only works with enwiki.

@dbarratt Oh cool! The end-result looks super similar to what I did in the end :D

Vvjjkkii renamed this task from Build a Docker container providing proxy access to Wiki Replicas to rqcaaaaaaa.Jul 1 2018, 1:09 AM
Vvjjkkii removed JeanFred as the assignee of this task.
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot renamed this task from rqcaaaaaaa to Build a Docker container providing proxy access to Wiki Replicas.Jul 1 2018, 7:06 PM
CommunityTechBot assigned this task to JeanFred.
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added a subscriber: Aklapper.

@JeanFred: Is there anything to follow up for this Hackathon session, or can this task be resolved? Thanks!

@JeanFred We actually found that most developers didn't want to remove the password from their ssh keys (which was required for running it in a container), so we ended up connecting to the SSH tunnel on the host:
https://github.com/wikimedia/InteractionTimeline/blob/master/.env.dist#L3-L5
here are the new instructions:
https://github.com/wikimedia/InteractionTimeline#contributing
the only problem we had was that on Linux, there is no host.docker.internal
https://github.com/docker/for-linux/issues/264
so we implemented a workaround to add the hostname on Linux:
https://github.com/wikimedia/InteractionTimeline/blob/master/bin/start#L5-L10

@JeanFred We actually found that most developers didn't want to remove the password from their ssh keys (which was required for running it in a container)

My SSH key does have a password, and I do run it in the container − that’s the whole point of mounting the SSH agent.

Ah!

This may not work on OSX, and definitely not on Windows

Yeah that was our problem, we needed a cross-platform solution. :/

@JeanFred: Has this happened at the Hackathon and should this task be closed as resolved? If not, which (active) project (tag) is this about?

@JeanFred: Has this happened at the Hackathon and should this task be closed as resolved? If not, which (active) project (tag) is this about?

Bstorm added a subscriber: Bstorm.

Because this seems long resolved, I'm going to go ahead and close this. It can always be reopened.