Page MenuHomePhabricator

add subbu to parsoid-roots
Closed, ResolvedPublic

Description

Can I be added to parsoid-roots? At this time, Roan is the only one in that group.

Details

Related Gerrit Patches:

Event Timeline

ssastry created this task.Jan 28 2016, 11:38 PM
ssastry raised the priority of this task from to Needs Triage.
ssastry updated the task description. (Show Details)
ssastry added a project: SRE-Access-Requests.
ssastry added a subscriber: ssastry.
Restricted Application added a project: Operations. · View Herald TranscriptJan 28 2016, 11:38 PM
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald Transcript
ssastry updated the task description. (Show Details)Jan 28 2016, 11:40 PM
ssastry set Security to None.
Dzahn renamed this task from Addition to parsoid-roots to add subbu to parsoid-roots .Jan 28 2016, 11:45 PM
RobH added a subscriber: RobH.Jan 29 2016, 4:52 PM

@ssastry: Since you already have shell, L3, etc... there are two things needed for this:

1.) Your managers approval for this access request expansion.
2.) Ops meeting review and approval for sudo rights.

Please have your manager approve on this task, and I'll ensure it is listed on next Monday's (2016-02-01) Ops meeting.

RobH added a comment.Jan 29 2016, 4:53 PM

I take that back, I actually don't see your signature on the L3 document either? (it is new and your access predates it, so that is not unusual.)

Would you additionally review and sign the document? It is simply an updated version of the agreement from back when your shell access was granted.

Dzahn added a subscriber: Dzahn.Jan 30 2016, 12:46 AM

Just to clarify, is this for all parsoid servers, incl. production or just for the test server ruthenium?

ssastry added a comment.EditedJan 30 2016, 5:20 AM

Just to clarify, is this for all parsoid servers, incl. production or just for the test server ruthenium?

Good question. Long answer, but I figured I would lay this out in detail so you have the full picture to help you evaluate the right solution from your end.

This access question came up because of having to do a whole bunch of preparatory work for setting up all the testing setup on ruthenium, not all of which can be done through selective sudo access, and I had to ping roots, and y'day have Tim build a package for me (debianizing that package would have been one solution, but I am not familiar with it, it felt premature, and felt like more overhead than necessary for what I was trying to accomplish). At that time, Tim mentioned that there was the parsoid-roots group which only had Roan on it. I gathered from Tim that you felt that having only Roan on the parsoid-roots group was a SPOF and that I should apply for parsoid-roots group. That is how this ticket came about.

All that said, in my case, I don't care for whether I have root access or not. I care more about being able to get through some of this test setup preparation quickly with low friction. This is going to happen again with T116090: Dedicated server for running Parsoid's roundtrip tests to get reliable parse latencies and use as perf. benchmarking tests when I start offloading some of the test clients there and where I care about consistent performance numbers. So, overall, I want access that lets me do this work quickly. I don't necessarily need it for the production parsoid servers. So far, there has been some only one relatively minor scenario that would have benefited from such access - killing stuck processes during parsoid deploys - but that is maybe once every 2 months, and ops-on-duty have handled it for me, so that is not a good enough reason for getting full root access to those servers. So, I'll let you guys weigh your policies and my use cases and figure out the right solution - i.e. do you want me to have access to those servers for other reasons?

But, I do care more about access to ruthenium at this time, and later for whatever server happens to get provisioned for T116090. To help you plan, I am going to lay out the 3 testing setups that I want to have in place over the next few weeks:

  1. Parsoid roundtrip testing: Run roundtrip tests on 160K titles. Right now, this takes about 16+ hours to complete. Once T116090 gets resolved, by using that only for test clients, we could cut this time down to 8 hours or so. This testing is required for us to verify that our Parsoid deployments won't cause (any known) regressions. Parsoid supports Flow, VE, and Content Translation, and reliability of those services depends on the reliability of our deployments. This is where this testing mode comes in.
  1. Parsoid <-> PHP-parser visual diff testing: We want to be able to run visual-diff tests on few 10s of thousand titles that takes screenshots of Parsoid HTML and PHP parser HTML, do a visual diff of those screenshots, and record a metric. This testing will let us make progress towards using Parsoid HTML as read views. This is a compute-intensive testing mode.
  1. Production Mediawiki <-> Test-Mediawiki visual diff testing: For scenarios like T89331: Replace HTML4 Tidy in MW parser with an equivalent HTML5 based tool, we need to be able to accurately evaluate the impact of those changes. We also want to be able to use this setup to evaluate changes to wikitext and how it impacts the wikipedia corpus. There are a bunch of related proposals down the pipeline for which we want to be able to reliably and authoritatively answer questions like: "what would happen if we change this piece of wikitext parsing in this way?" Just like 2. above, we want to be able to run visual diff tests on a few 10s of thousands titles. This is a compute-intensive test mode.

Given the compute-intensive nature of all these testing involved, have them all run on ruthenium is not possible. So, as we acquire other servers (either other hardware as part of T116090, labs vms, or labs-physical-hardware), there is some amount of test setup experimentation that is going to be involved here. Having to ping roots for every step seems not very viable while I am doing this. Also, puppetizing every little step seems like overkill given that some of this is probably a one-off setup step. But, I am committed to puppetizing as much of this process as possible, but once, I figure out solutions for where it makes sense. If any of the servers involved in these test setups go down, that will help minimize disruption. But, in this process of setting up, having full access to whatever servers are involved would be greatly helpful. To re-iterate, it is not root access I am after, but being able to do this work efficiently without complicating your work in the process. It seemed like getting on parsoid-roots was the simplest way of accomplishing it.

Hope this helps you all in figuring out the right level and right mode of access. Feel free to ask me other questions or request additional details.

I approve giving Subbu access to ruthenium.

In addition to what Trevor approved above, based on ops recommendations, I can request and get @TrevorParscal's approval for the same.

RobH awarded a token.Feb 1 2016, 8:28 PM
RobH added a subscriber: mark.Feb 1 2016, 8:49 PM
This comment was removed by RobH.
ssastry triaged this task as High priority.Feb 1 2016, 8:50 PM
RobH added a comment.Feb 1 2016, 8:55 PM

My last comment (and token) was meant for an unrelated task i had open in another tab. I removed the comment but tokens seem to stick (sorry about that!)

Krenair rescinded a token.
elukey added a subscriber: elukey.Feb 3 2016, 5:04 PM
mark added a comment.Feb 3 2016, 5:05 PM

Let's do this - Approved.

(I'll get to the other tasks tomorrow as soon as I get out of budget frenzy ;)

elukey added a comment.EditedFeb 3 2016, 5:53 PM

As discussed with Ops, we don't want to block Subramanya and I just made the change to add a new group to ruthenium:

https://gerrit.wikimedia.org/r/#/c/268167/

elukey@ruthenium:~$
...
Notice: /Stage[main]/Admin/Admin::Hashgroup[parsoid-test-roots]/Admin::Group[parsoid-test-roots]/Group[parsoid-test-roots]/ensure: created
...

Thanks to Dzahn for the guidelines :)

Dzahn assigned this task to elukey.Feb 3 2016, 6:10 PM

@ssastry you got root

@ruthenium:~# id ssastry
uid=2316(ssastry) gid=500(wikidev) groups=500(wikidev),702(parsoid-admin),772(parsoid-test-roots)

@ruthenium:~# cat /etc/sudoers.d/parsoid-test-roots 
# This file is managed by Puppet!

%parsoid-test-roots ALL = (ALL) NOPASSWD: ALL
Dzahn closed this task as Resolved.Feb 3 2016, 6:10 PM

Change 269526 had a related patch set uploaded (by Andrew Bogott):
Move promethium into wikitechexp so subbu can use it.

https://gerrit.wikimedia.org/r/269526

Change 269526 merged by Andrew Bogott:
Move promethium into wikitechexp so subbu can use it.

https://gerrit.wikimedia.org/r/269526