Page MenuHomePhabricator

Give all members of the Parsing team production `deployment` access ( add arlolra to deployers)
Closed, ResolvedPublic

Description

Needed for the new world of Parsoid-in-production-as-part-of-MW.

They all currently have parsoid-admin (except sbailey). deployment is needed to run scap in /srv/mediawiki-staging/, which the new plan requires.

James F. will give them a crash-course on how MW-land production deployments work.

Event Timeline

Restricted Application added a project: Operations. · View Herald TranscriptFeb 21 2020, 10:18 PM
Jdforrester-WMF renamed this task from Give all members of the Parsing team production deployment access to Give all members of the Parsing team production `deployment` access.Feb 21 2020, 10:19 PM
Jdforrester-WMF updated the task description. (Show Details)
jbond triaged this task as Medium priority.Feb 24 2020, 1:11 PM
Dzahn updated the task description. (Show Details)Feb 25 2020, 9:59 PM
Dzahn added subscribers: Sbailey, cscott, Arlolra.

Hi all,

those of you who have not signed it yet, please read and sign L3.

@Sbailey Please create a new SSH keypair and paste the public part here on the ticket. See https://wikitech.wikimedia.org/wiki/Production_access#Generating_your_SSH_key

Adding @greg for deployment access oversight.

@ssastry @Jdforrester-WMF Should sbailey be added to the parsoid-admins group as well so that all people who can deploy can also get shell access?

those of you who have not signed it yet, please read and sign L3.

Looks like I signed it in 2016.

@ssastry @Jdforrester-WMF Should sbailey be added to the parsoid-admins group as well so that all people who can deploy can also get shell access?

I assume without shell access, @Sbailey cannot log on to scandium, correct? If so, yes, please add to the parsoid-admins group.

@ssastry Yea, though it's not tied to the hostname. It's "the puppet role parsoid::testing installs the admin groups parsoid-roots, parsoid-admin, parsoid-test-admins and parsoid-test-roots. But if she is in none of these groups she won't get shell access on scandium. Correct. Which of the 4 groups is the most appropriate and why we put the non-test admins also on the test machine while having separate test-only admin groups is another question.

@ssastry Yea, though it's not tied to the hostname. It's "the puppet role parsoid::testing installs the admin groups parsoid-roots, parsoid-admin, parsoid-test-admins and parsoid-test-roots. But if she is in none of these groups she won't get shell access on scandium. Correct. Which of the 4 groups is the most appropriate and why we put the non-test admins also on the test machine while having separate test-only admin groups is another question.

@Sbailey and I talked about this, and arrived at the following:

Let us start with granting @Sbailey access to just the testing roles, so, parsoid-test-admins (or roots as appropriate). She needs to be able to login to that server and kick off new tests, set up ssh tunnels to look at rt testing results, etc. Let us grant her deployment rights and shell access to the production cluster in a later step once she feels comfortable.

So, @cscott said he has signed the form as well. @Arlolra is out but is possible he has signed the form previously as well. Can you check and if you have all the necessary information, you can proceed with the next steps. Thanks!

Change 575097 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] admins: add Shannon Bailey to parsoid-test groups, upgrade to shell user

https://gerrit.wikimedia.org/r/575097

Dzahn added a comment.EditedFeb 26 2020, 10:45 PM

@Sbailey @ssastry Sounds good. Now we just need an SSH public key from you. I started a patch to upgrade you from ldap_only admin to shell admin but need to amend it with your key.

P.S. only adding to test-roots because test-admins is a more limited subset of that.

Let us start with granting @Sbailey access to just the testing roles, so, parsoid-test-admins (or roots as appropriate). She needs to be able to login to that server and kick off new tests, set up ssh tunnels to look at rt testing results, etc. Let us grant her deployment rights and shell access to the production cluster in a later step once she feels comfortable.

ACK, using test-roots. So far that is just you and adding another person sounds like a good idea either way.

So, @cscott said he has signed the form as well. @Arlolra is out but is possible he has signed the form previously as well. Can you check and if you have all the necessary information, you can proceed with the next steps. Thanks!

Confirmed signatures from ssastry, cscott, arlolra .. ACK.

greg added a comment.Feb 27 2020, 7:50 PM

Approved from my end.

Change 575393 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] admins: adding arlolra to deployers

https://gerrit.wikimedia.org/r/575393

Dzahn added a comment.EditedFeb 28 2020, 1:53 AM

@cscott @ssastry Actually you already have deployment access. So this is just about adding arlolra now.

I note also @ppchelko has already deployment. Excluding mobrovac this means all parsoid-admins will have deployment as well.

[deploy1001:~] $ for user in cscott ssastry arlolra ppchelko sbailey; do id $user; done
uid=2880(cscott) gid=500(wikidev) groups=500(wikidev),705(deployment),763(deploy-service),702(parsoid-admin)
uid=2316(ssastry) gid=500(wikidev) groups=500(wikidev),705(deployment),763(deploy-service),702(parsoid-admin)
uid=3381(arlolra) gid=500(wikidev) groups=500(wikidev),763(deploy-service),702(parsoid-admin)
uid=12460(ppchelko) gid=500(wikidev) groups=500(wikidev),705(deployment),763(deploy-service),702(parsoid-admin)
id: ‘sbailey’: no such user
Dzahn renamed this task from Give all members of the Parsing team production `deployment` access to Give all members of the Parsing team production `deployment` access ( add arlolra to deployers).Feb 28 2020, 1:54 AM

Change 575393 merged by Dzahn:
[operations/puppet@production] admins: adding arlolra to deployers

https://gerrit.wikimedia.org/r/575393

Dzahn added a comment.Feb 28 2020, 1:59 AM

@Arlolra Has been added to deployers. This would solve this ticket now except of the side-task to add sbailey to test-groups. We still need the SSH key for that please.

Dzahn assigned this task to Sbailey.Feb 28 2020, 2:00 AM

@Arlolra Has been added to deployers.

Thank you

Greg, do you really want me to post my public key in this form? or send it in a seperate message?

Subbu says yes to here it is:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCYenmO37Azx5fr7WO5F107ZL6g3o5AAgkLsGtheyCC9soguCltps4a2MBrw3nBMVXRPFSfJiTvsWARUC40d0oWxcqpXkRtHrY4mNzMMMz8lFzxRQ8ts+zKdU2KZJ8vaEhCnVmRemvIGE8xlaVPEleaAeyAZOjKxBaPVHB9CX7b0QgnZ+RTZRXi9BD1Aa0gnYurwJ5tA1nwPub0MRtGrkXhVaG78rUpkHECeM5BbmQoZnrvL4J9FdRkMkifEyHop+75RweUNbFhkiQ6pNia3MSMBC2tSYSL8IaLCqha8lFDkwHBNpzG09mpzFw1La6Niuth3CQ/72U9dQRvQFk+/Mp shannonbailey@wmf1287.local

Dzahn added a comment.Feb 28 2020, 7:10 PM

Thanks @Sbailey yes, that's correct. We actually want it on the ticket and that is public info either way. I'll go right ahead now.

Dzahn claimed this task.Feb 28 2020, 7:10 PM

Change 575599 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] admins: upgrade Shannon Bailey from ldap_only to shell, parsoid-test-root

https://gerrit.wikimedia.org/r/575599

Change 575599 merged by Dzahn:
[operations/puppet@production] admins: upgrade Shannon Bailey from ldap_only to shell, parsoid-test-root

https://gerrit.wikimedia.org/r/575599

Dzahn closed this task as Resolved.Feb 28 2020, 8:21 PM

@Sbailey You now have root access to the parsoid-testing servers. Currently this means only scandium.eqiad.wmnet, but in the future it could be other host names. Your access will automatically move along with the parsoid::testing role.

[scandium:~] $ id sbailey
uid=18035(sbailey) gid=500(wikidev) groups=500(wikidev),772(parsoid-test-roots)
Dzahn added a comment.Feb 28 2020, 8:22 PM

@Sbailey Here you see an example SSH config for jumping via bastion hosts to other servers (scandium) behind it. https://wikitech.wikimedia.org/wiki/Production_access#Setting_up_your_SSH_config

I'm sure others in your team with existing access also can you walk through it.

Change 575097 abandoned by Dzahn:
admins: add Shannon Bailey to parsoid-test groups, upgrade to shell user

Reason:
duplicate of https://gerrit.wikimedia.org/r/c/operations/puppet/ /575599

https://gerrit.wikimedia.org/r/575097

Dzahn added a comment.Feb 28 2020, 8:48 PM

@Arlolra and @cscott You also have root on parsoid::testing (scandium) now, alongside sbailey.

cscott added a comment.Mar 3 2020, 1:02 AM

Ok, tested deploy rights today. We depooled wtp1025 temporarily for testing. On wtp1025 I had:

cscott@wtp1025:/srv/mediawiki$ groups
wikidev parsoid-admin

and when I tried:

cscott@wtp1025:/srv/mediawiki$ scap pull
00:03:57 Copying from deploy1001.eqiad.wmnet to wtp1025.eqiad.wmnet
00:03:57 Started rsync common
sudo: a password is required

...followed by miscellaneous errors. looks like either i don't have the required perms or my sudo setup is weird.

@Jdforrester-WMF is in 'wikidev deployment deploy-service`.

So it looks like parsoid-admin isn't enough...

Dzahn reopened this task as Open.Mar 3 2020, 2:10 PM
Dzahn added a comment.Mar 3 2020, 3:21 PM

Running scap from the deployment server (what James does) and running scap pull on a single host directly require different admin groups and permissions. Yea, parsoid-admin is currently just about restarting the parsoid and parsoid-rt-client services but unrelated to deployment and scap.

Change 576383 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] admins: let parsoid-admins run scap pull as mwdeploy

https://gerrit.wikimedia.org/r/576383

Dzahn added a comment.Mar 3 2020, 4:43 PM

The change above would let parsoid-admins run "scap pull" AS user "mwdeploy". I tested that as user cscott on wtp1025 and it worked for me. Without a new rule like this i can confirm the errors described above when running scap pull as cscott. If i run 'scap pull" (without typing sudo) as my own user i don't have errors. That's because i'm in the 'ops' group which has sudo ALL ALL.

Change 576383 merged by Dzahn:
[operations/puppet@production] admins: let parsoid-admins run scap pull as mwdeploy

https://gerrit.wikimedia.org/r/576383

Dzahn added a comment.Mar 3 2020, 5:23 PM

@cscott @ssastry After the merge above and the next puppet run all existing "parsoid-admins" should now be able to run scap pull _as mwdeploy_ on all wtp* hosts.

So use sudo -u mwdeploy scap pull and it should work without errors.

Dzahn closed this task as Resolved.Mar 3 2020, 6:07 PM