Can I be added to parsoid-roots? At this time, Roan is the only one in that group.
|operations/puppet : production||Move promethium into wikitechexp so subbu can use it.|
- Mentioned In
- T124701: Getting parsing-team members sudo access to manage (start, stop, restart) services on ruthenium
- Mentioned Here
- T89331: Replace HTML4 Tidy in MW parser with an equivalent HTML5 based tool
T116090: Dedicated server for running Parsoid's roundtrip tests to get reliable parse latencies and use as perf. benchmarking tests
@ssastry: Since you already have shell, L3, etc... there are two things needed for this:
1.) Your managers approval for this access request expansion.
2.) Ops meeting review and approval for sudo rights.
Please have your manager approve on this task, and I'll ensure it is listed on next Monday's (2016-02-01) Ops meeting.
I take that back, I actually don't see your signature on the L3 document either? (it is new and your access predates it, so that is not unusual.)
Would you additionally review and sign the document? It is simply an updated version of the agreement from back when your shell access was granted.
Good question. Long answer, but I figured I would lay this out in detail so you have the full picture to help you evaluate the right solution from your end.
This access question came up because of having to do a whole bunch of preparatory work for setting up all the testing setup on ruthenium, not all of which can be done through selective sudo access, and I had to ping roots, and y'day have Tim build a package for me (debianizing that package would have been one solution, but I am not familiar with it, it felt premature, and felt like more overhead than necessary for what I was trying to accomplish). At that time, Tim mentioned that there was the parsoid-roots group which only had Roan on it. I gathered from Tim that you felt that having only Roan on the parsoid-roots group was a SPOF and that I should apply for parsoid-roots group. That is how this ticket came about.
All that said, in my case, I don't care for whether I have root access or not. I care more about being able to get through some of this test setup preparation quickly with low friction. This is going to happen again with T116090: Dedicated server for running Parsoid's roundtrip tests to get reliable parse latencies and use as perf. benchmarking tests when I start offloading some of the test clients there and where I care about consistent performance numbers. So, overall, I want access that lets me do this work quickly. I don't necessarily need it for the production parsoid servers. So far, there has been some only one relatively minor scenario that would have benefited from such access - killing stuck processes during parsoid deploys - but that is maybe once every 2 months, and ops-on-duty have handled it for me, so that is not a good enough reason for getting full root access to those servers. So, I'll let you guys weigh your policies and my use cases and figure out the right solution - i.e. do you want me to have access to those servers for other reasons?
But, I do care more about access to ruthenium at this time, and later for whatever server happens to get provisioned for T116090. To help you plan, I am going to lay out the 3 testing setups that I want to have in place over the next few weeks:
- Parsoid roundtrip testing: Run roundtrip tests on 160K titles. Right now, this takes about 16+ hours to complete. Once T116090 gets resolved, by using that only for test clients, we could cut this time down to 8 hours or so. This testing is required for us to verify that our Parsoid deployments won't cause (any known) regressions. Parsoid supports Flow, VE, and Content Translation, and reliability of those services depends on the reliability of our deployments. This is where this testing mode comes in.
- Parsoid <-> PHP-parser visual diff testing: We want to be able to run visual-diff tests on few 10s of thousand titles that takes screenshots of Parsoid HTML and PHP parser HTML, do a visual diff of those screenshots, and record a metric. This testing will let us make progress towards using Parsoid HTML as read views. This is a compute-intensive testing mode.
- Production Mediawiki <-> Test-Mediawiki visual diff testing: For scenarios like T89331: Replace HTML4 Tidy in MW parser with an equivalent HTML5 based tool, we need to be able to accurately evaluate the impact of those changes. We also want to be able to use this setup to evaluate changes to wikitext and how it impacts the wikipedia corpus. There are a bunch of related proposals down the pipeline for which we want to be able to reliably and authoritatively answer questions like: "what would happen if we change this piece of wikitext parsing in this way?" Just like 2. above, we want to be able to run visual diff tests on a few 10s of thousands titles. This is a compute-intensive test mode.
Given the compute-intensive nature of all these testing involved, have them all run on ruthenium is not possible. So, as we acquire other servers (either other hardware as part of T116090, labs vms, or labs-physical-hardware), there is some amount of test setup experimentation that is going to be involved here. Having to ping roots for every step seems not very viable while I am doing this. Also, puppetizing every little step seems like overkill given that some of this is probably a one-off setup step. But, I am committed to puppetizing as much of this process as possible, but once, I figure out solutions for where it makes sense. If any of the servers involved in these test setups go down, that will help minimize disruption. But, in this process of setting up, having full access to whatever servers are involved would be greatly helpful. To re-iterate, it is not root access I am after, but being able to do this work efficiently without complicating your work in the process. It seemed like getting on parsoid-roots was the simplest way of accomplishing it.
Hope this helps you all in figuring out the right level and right mode of access. Feel free to ask me other questions or request additional details.
As discussed with Ops, we don't want to block Subramanya and I just made the change to add a new group to ruthenium:
Notice: /Stage[main]/Admin/Admin::Hashgroup[parsoid-test-roots]/Admin::Group[parsoid-test-roots]/Group[parsoid-test-roots]/ensure: created
Thanks to Dzahn for the guidelines :)