Page MenuHomePhabricator

Requesting access to maps for mbsantos and jgiannelos
Closed, ResolvedPublicRequest

Description

Requestor provided information and prerequisites

The users already have some level of access to the maps cluster. What we need here is permission to use a depooled machine maps2007.codfw.wmnet to test new puppet rules. How can we proceed with that?

Reason

There are a few blockers:

  • The tool imposm3 is not on debian stable channel, so to install it on the server we need to backport the binary
  • The OSM sync uses scripts that interacts with DB in 2 different levels: import and update. So we need to be careful with the permissions we are giving to the scripts
  • We can manually install imposm3 on beta-cluster machines and test some of the scripts, but testing on beta cluster is good at a certain level, we can’t test full planet data because of disk space

We need the right level of permissions to be testing the new puppet rules at maps2007.codfw.wmnet before being ready for production and review from SRE.

This section is to be completed by the individual requesting access.

  • Wikitech username:
  • Preferred shell username:
  • Email address:
  • Ssh public key (must be dedicated key for wmf production):
  • Requested group membership:
  • Reason for access:
  • Name of approving party (hiring manager for WMF staff):
  • Requestor -- Please Acknowledge that you have read and signed the L3 Wikimedia Server Access Responsibilities document:
  • Requestor -- Please coordinate obtaining a comment of approval on this task from the approving party.

SRE Clinic Duty Confirmation Checklist for Access Requests

This checklist should be used on all access requests to ensure that all steps are covered, including expansion to existing access. Please double check the step has been completed before checking it off.

This section is to be confirmed and completed by a member of the SRE team.

  • - User has signed the L3 Acknowledgement of Wikimedia Server Access Responsibilities Document.
  • - User has a valid NDA on file with WMF legal. (This can be checked by Operations via the NDA tracking sheet & is included in all WMF Staff/Contractor hiring.)
  • - User has provided the following: wikitech username, preferred shell username, email address, and full reasoning for access (including what commands and/or tasks they expect to perform)
  • - User has provided a public SSH key. This ssh key pair should only be used for WMF cluster access, and not share with any other service (this includes not sharing with WMCS access, no shared keys.)
  • - access request (or expansion) has sign off of WMF sponsor/manager (sponser for volunteers, manager for wmf staff)
  • - non-staff requests: 3 business day wait must pass with no objections being noted on the task
  • - Patchset for access request

For additional details regarding access request requirements, please see https://wikitech.wikimedia.org/wiki/Requesting_shell_access

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Is this specifically only about maps2007? Given that you're both members of the maps-admin group you can log into any maps host, but maps2007 in particular is currently inaccessible since Puppet was disabled there for a long time (which made it evicted from the system records in the Puppet database). This needs to be fixed in general and is unrelated to your access permissions, right now not even people in SRE can perform a standard SSH login into the system in the current state of it.

Is this specifically only about maps2007? Given that you're both members of the maps-admin group you can log into any maps host, but maps2007 in particular is currently inaccessible since Puppet was disabled there for a long time (which made it evicted from the system records in the Puppet database). This needs to be fixed in general and is unrelated to your access permissions, right now not even people in SRE can perform a standard SSH login into the system in the current state of it.

@MoritzMuehlenhoff hmm, I assumed that the machine was only out of production traffic. I guess there is another machine depooled (maps2002) but this machine have disk space issues with Cassandra.

Is this specifically only about maps2007? Given that you're both members of the maps-admin group you can log into any maps host, but maps2007 in particular is currently inaccessible since Puppet was disabled there for a long time (which made it evicted from the system records in the Puppet database). This needs to be fixed in general and is unrelated to your access permissions, right now not even people in SRE can perform a standard SSH login into the system in the current state of it.

@MoritzMuehlenhoff hmm, I assumed that the machine was only out of production traffic. I guess there is another machine depooled (maps2002) but this machine have disk space issues with Cassandra.

The reason for the disabling is not yet clustered per puppet, which suggests to me that it's more than just out of rotation for incoming production traffic. maps2002 is in a similar (?) state. No puppet runs since a month and disabling is rebuilding cassandra.

As far as imposm3 goes, maps is on stretch, not even debian stable, but rather oldstable. However it does look like we can get it build and shipped easily and I 'd much rather prefer that since that way we would not end up populating versionless binaries on the production hosts, losing auditing. Assuming we get the package easily installed, what kind of rights would you need ?

I am also interested in is the to be testing the new puppet rules part. Could you please share a bit on how this will be done?

Is this specifically only about maps2007? Given that you're both members of the maps-admin group you can log into any maps host, but maps2007 in particular is currently inaccessible since Puppet was disabled there for a long time (which made it evicted from the system records in the Puppet database). This needs to be fixed in general and is unrelated to your access permissions, right now not even people in SRE can perform a standard SSH login into the system in the current state of it.

@MoritzMuehlenhoff hmm, I assumed that the machine was only out of production traffic. I guess there is another machine depooled (maps2002) but this machine have disk space issues with Cassandra.

The reason for the disabling is not yet clustered per puppet, which suggests to me that it's more than just out of rotation for incoming production traffic. maps2002 is in a similar (?) state. No puppet runs since a month and disabling is rebuilding cassandra.

maps2002 is with a disk space issue caused by Cassandra replication, that's why it's depooled.

As far as imposm3 goes, maps is on stretch, not even debian stable, but rather oldstable. However it does look like we can get it build and shipped easily and I 'd much rather prefer that since that way we would not end up populating versionless binaries on the production hosts, losing auditing. Assuming we get the package easily installed, what kind of rights would you need ?

Yeah, AFAIK imposm in every debian channel is pinned to imposm2. imposm3 complete rewrite in golang and since it was released, never got into debian channels

I am also interested in is the to be testing the new puppet rules part. Could you please share a bit on how this will be done?

If the only way to apply puppet rules is through the master branch, I'm thinking of a conditional rule where it would load the new imposm_planet_sync classes. And then I would add the needed parameters to the hieradata configuration for the machine available for test.

Regarding the packaging part, here is some context about the state of imposm3 and debian: https://phabricator.wikimedia.org/T238753
Just to add to what Mateus said, our ideas was to investigate how complicated would be to build a debian package for our needs or at least just for the interim for testing purposes.

I am also interested in is the to be testing the new puppet rules part. Could you please share a bit on how this will be done?

If the only way to apply puppet rules is through the master branch, I'm thinking of a conditional rule where it would load the new imposm_planet_sync classes. And then I would add the needed parameters to the hieradata configuration for the machine available for test.

Yes, unfortunately the master (named production for historical reasons) branch is more or less the only thing that makes sense in our setup. So if the tests/changes are as scoped in nature as the example above, yes let's go with that, it will work fine.

Regarding the packaging part, here is some context about the state of imposm3 and debian: https://phabricator.wikimedia.org/T238753
Just to add to what Mateus said, our ideas was to investigate how complicated would be to build a debian package for our needs or at least just for the interim for testing purposes.

Yup, I 've been commenting there already. There's a couple of paths forward, it might take a while to figure out exactly what we want.

jbond added subscribers: hnowlan, jbond.

Im going to drop the SRE-Access-Requests tage from this task as i dosn't look like there is an access request to action. please re add if i have missed something.

Dzahn changed the task status from Open to Stalled.Feb 10 2021, 8:01 PM

@MSantos This is request is almost 2 years old, is it still relevant?

LSobanski lowered the priority of this task from High to Medium.Nov 7 2022, 10:21 AM
MoritzMuehlenhoff claimed this task.

@Jgiannelos and @MSantos were added to the maps-root group back in September 2021, closing this task.