Page MenuHomePhabricator

Volans (Riccardo Coccioli)
SRE

Projects (12)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.

User Details

User Since
Feb 10 2016, 11:25 AM (274 w, 1 d)
Availability
Available
IRC Nick
volans
LDAP User
Volans
MediaWiki User
RCoccioli (WMF) [ Global Accounts ]

Recent Activity

Today

Volans added a comment to T271583: Implement early-racking automation (was: Productionize system to automatically deploy some BIOS Settings).

Great! Thanks a lot!

Thu, May 13, 4:40 PM · User-crusnov, Patch-For-Review, DC-Ops, SRE-tools
Volans added a comment to T282787: Configure dns and puppet repositories for new drmrs datacenter.

Other random things that needs to be updated sooner or later. I hope you don't mind if I drop them here, feel free to move them to a dedicated task.

Thu, May 13, 2:50 PM · SRE, Traffic
herron awarded T282784: New VictorOps user request a Party Time token.
Thu, May 13, 2:26 PM · SRE, SRE-Access-Requests, observability
Volans added a comment to T282784: New VictorOps user request.

@cmooney I've invited you and added you to the SRE team. Please follow the instructions at https://wikitech.wikimedia.org/wiki/VictorOps#Set_up_as_a_new_user

Thu, May 13, 1:54 PM · SRE, SRE-Access-Requests, observability
Volans claimed T282784: New VictorOps user request.
Thu, May 13, 1:52 PM · SRE, SRE-Access-Requests, observability
Volans edited Description on SRE.
Thu, May 13, 11:27 AM
Volans added a comment to T282775: Revert workaround for cumin output verbosity on RemoteExecution (CuminExecution) abstraction.

Version is 4.1.0, currently latest on apt.w.o and PyPI and deployed to all cumin hosts in production.
See worker.reporter and worker.progress_bars on https://doc.wikimedia.org/cumin/master/introduction.html#library for an example of how to implement it.

Thu, May 13, 11:02 AM · SRE-tools, Data-Persistence-Backup, DBA

Yesterday

Volans closed T270795: cumin's test dependencies trigger excessive pip backtracking as Resolved.

Cumin 4.1.0 has been released to PyPI, resolving. Feel free to re-open in case there is still any issue.

Wed, May 12, 10:23 PM · SRE-tools
Volans closed T281314: Decom cookbook should also search deployment-charts for host references as Resolved.
Wed, May 12, 5:07 PM · SRE-tools
Volans added a comment to T281314: Decom cookbook should also search deployment-charts for host references.

The patch that add the feature has been merged and deployed, resolving. I don't have any host to decommission right now, but I tested the dry-run mode and run manually the specific command that the cookbook would run and seems to work fine.
Feel free to reopen if you encounter any issue.

Wed, May 12, 5:07 PM · SRE-tools
Volans triaged T281314: Decom cookbook should also search deployment-charts for host references as Medium priority.
Wed, May 12, 1:29 PM · SRE-tools
Volans committed rOSHPd50086a160a5: Update debian bullseye frozen requirements (authored by Volans).
Update debian bullseye frozen requirements
Wed, May 12, 12:16 PM
Volans committed rOSHP080c82a533bd: Build artifacts for Debian bullseye (authored by Volans).
Build artifacts for Debian bullseye
Wed, May 12, 10:10 AM

Tue, May 11

aborrero awarded T282529: debmonitor: 500 on upgraded+downgraded package a Love token.
Tue, May 11, 3:21 PM · SRE-tools
Volans added a comment to T282529: debmonitor: 500 on upgraded+downgraded package.

The above patch fixes the issue. For reference this is what's in apt cache for this specific package.

Tue, May 11, 2:16 PM · SRE-tools
Volans added a comment to T282529: debmonitor: 500 on upgraded+downgraded package.

So, after some digging the issue here is that the source package reported by debmonitor-client is wrong because taken from the 'candidate' package and not the actual version to be installed (candidate is true in almost all cases in our infra) that in this case was specified manually.
I'm sending a patch to fix the client so that it correctly detects the right source package.

Tue, May 11, 1:24 PM · SRE-tools
Volans added a comment to T282529: debmonitor: 500 on upgraded+downgraded package.

I think I know what's happening here, I double check a couple of things and update the task accordingly later.

Tue, May 11, 10:52 AM · SRE-tools
Volans added a comment to T282529: debmonitor: 500 on upgraded+downgraded package.

Applying the above patch as a quick hot-patch I was able to get a full stacktrace:

[2021-05-11T10:04:11] Unable to update host 'cloudgw2001-dev.codfw.wmnet'
Traceback (most recent call last):
  File "/srv/deployment/debmonitor/venv/lib/python3.7/site-packages/django/db/models/query.py", line 486, in get_or_create
    return self.get(**lookup), False
  File "/srv/deployment/debmonitor/venv/lib/python3.7/site-packages/django/db/models/query.py", line 399, in get
    self.model._meta.object_name
bin_packages.models.PackageVersion.DoesNotExist: PackageVersion matching query does not exist.
Tue, May 11, 10:41 AM · SRE-tools
Volans triaged T282529: debmonitor: 500 on upgraded+downgraded package as High priority.
Tue, May 11, 9:31 AM · SRE-tools

Mon, May 10

Volans added a comment to T281314: Decom cookbook should also search deployment-charts for host references.

@Legoktm sure, why not. Is there a canonical place where the repository is checkout at the commit that is currently in production?

Mon, May 10, 3:20 PM · SRE-tools
Volans committed rOHPU58d3ce1b3e2a: firewall: add cumin2002 to the cumin term (authored by Volans).
firewall: add cumin2002 to the cumin term
Mon, May 10, 9:05 AM
Volans closed T271827: Puppet broken in the automation-framework project as Resolved.

I've deleted all the old instances that included also all the ones that had puppet broken. Sorry for the delay.

Mon, May 10, 8:25 AM · Cloud-VPS, SRE-tools

Thu, May 6

Dzahn awarded T277740: Support downtiming services in our cookbooks a Love token.
Thu, May 6, 6:39 PM · Prod-Kubernetes, SRE, SRE-tools
Volans closed T277740: Support downtiming services in our cookbooks as Resolved.

@JMeybohm this is now all supported.
We have a sre.hosts.remove-downtime cookbook that when run with --force will ask the user if it wants to proceed with the hosts without verifying them with puppetdb:

Thu, May 6, 6:05 PM · Prod-Kubernetes, SRE, SRE-tools
Volans closed T240176: OKR: Provide a mechanism to obtain the FQDN given a short hostname as Resolved.

This is now possible via spicerack in any cookbook using:

Thu, May 6, 5:31 PM · netbox, SRE-tools

Wed, May 5

Volans added a comment to T282019: sre.hosts.decommission: don't FAIL when unable to set icinga downtime.

I'm not sure if hiding it completely is a great choice. We could look to see if we could improve the check and distinguish between a missing host and other failures.

Sounds good, yeah I would want to know if it happens as well. Maybe we could consider the step optional? In other words still report on it, but its outcome wouldn't affect the overall PASS/FAIL status.

Wed, May 5, 7:32 PM · SRE-tools
Volans triaged T282019: sre.hosts.decommission: don't FAIL when unable to set icinga downtime as Medium priority.
Wed, May 5, 5:05 PM · SRE-tools
Volans added a comment to T282019: sre.hosts.decommission: don't FAIL when unable to set icinga downtime.

Thanks for the task, while it's true that it could have been already removed from Icinga, it might happen for other reasons too and I'm not sure if hiding it completely is a great choice. We could look to see if we could improve the check and distinguish between a missing host and other failures.
Do you happen to know why that step failed in this specific case? And if was that the host was already not anymore in Icinga, do you know why?
In the usual workflow a host to be decommissioned should be in Icinga.

Wed, May 5, 5:05 PM · SRE-tools
Volans committed rOSHP2e5336bbb84d: Use system python to define variable (authored by Volans).
Use system python to define variable
Wed, May 5, 2:12 PM
Volans added a comment to T281596: Publish wikimedia-bullseye base docker image.

If possible it would be great if we could get a base bullseye image somehow, even if not auto-updated right from the start and otherwise created.
We have already a couple of hosts on bullseye and I need to add a python-build bullseye image to the registry to build the wheels of a Python application included in the role of one of the hosts, that of course requires a base bullseye image.
FYI Moritz has opened T281984 for the long term solution.

Wed, May 5, 1:40 PM · SRE, serviceops

Mon, May 3

Volans committed rCUMIN29a45263e174: Upstream release v4.1.0 (authored by Volans).
Upstream release v4.1.0
Mon, May 3, 11:25 AM
Volans committed rCUMIN2dfdf98ee6d8: Merge tag 'tags/v4.1.0' into debian (authored by Volans).
Merge tag 'tags/v4.1.0' into debian
Mon, May 3, 11:08 AM
Volans committed rCUMIN8aa07029411d: CHANGELOG: add changelogs for release v4.1.0 (authored by Volans).
CHANGELOG: add changelogs for release v4.1.0
Mon, May 3, 11:01 AM
Volans committed rCUMIN354e1999e94f: doc: fix sphinx warning in docstring (authored by Volans).
doc: fix sphinx warning in docstring
Mon, May 3, 9:51 AM

Thu, Apr 29

Volans committed rCUMINd59eee85e01d: setup.py: add missing config (authored by Volans).
setup.py: add missing config
Thu, Apr 29, 2:04 PM
Volans committed rOBPY06b1d8d03458: Remove Python2 support, bullseye compatibility (authored by Volans).
Remove Python2 support, bullseye compatibility
Thu, Apr 29, 11:13 AM
Volans committed rOSHOb5618ea7869c: CHANGELOG: add changelogs for release v0.2.8 (authored by Volans).
CHANGELOG: add changelogs for release v0.2.8
Thu, Apr 29, 9:40 AM
Volans committed rOSHO36bf14af77f7: setup.py: limit max version of pynetbox (authored by Volans).
setup.py: limit max version of pynetbox
Thu, Apr 29, 9:40 AM
Volans committed rOSHPe394769e7b0c: Upstream release v0.2.8 (authored by Volans).
Upstream release v0.2.8
Thu, Apr 29, 9:30 AM
Volans committed rOSHP89cd07cda2b3: Upstream release v0.2.7 (authored by Volans).
Upstream release v0.2.7
Thu, Apr 29, 8:24 AM
Volans committed rOSNE832d558b1ab4: customscript: fix typo in interface automation (authored by Volans).
customscript: fix typo in interface automation
Thu, Apr 29, 8:19 AM
Volans committed rOSNE71d51f3ccddf: customscript: fix VIP assignment in PuppetDB import (authored by Volans).
customscript: fix VIP assignment in PuppetDB import
Thu, Apr 29, 7:58 AM
Volans added a comment to T281347: tox-docker CI test doesn't pick up overrides for pylint.

Also if you want to support bullseye it would be good to add 3.9 support to tox and setup.py too.

Thu, Apr 29, 7:53 AM · ci-test-error, SRE
Volans added a comment to T281347: tox-docker CI test doesn't pick up overrides for pylint.

In setup.py there is a very specific version of prospector from 2018 ('prospector[with_everything]==1.1.6.2').
This, in conjunction with the fact that the same virtualenv is used for all tox envs of a given Python version it means that the dependency tree must be compiled in such a way that satisfy all the constraints and I think this brings in turns older versions of many of the tools, although I didn't dig to prove it.

Thu, Apr 29, 7:52 AM · ci-test-error, SRE

Tue, Apr 27

Volans committed rCUMIN33a167cdc79a: setup.py: support more recent PyParsing versions (authored by Volans).
setup.py: support more recent PyParsing versions
Tue, Apr 27, 5:50 PM
Volans committed rCUMIN5d56e1133551: clustershell: instantiate progress bar earlier (authored by Volans).
clustershell: instantiate progress bar earlier
Tue, Apr 27, 5:50 PM
Volans committed rCUMINd7e90c7bf3bd: CLI/clustershell: allow to disable progress bars (authored by Volans).
CLI/clustershell: allow to disable progress bars
Tue, Apr 27, 5:50 PM
Volans committed rCUMIN1fcb359141ea: clustershell: allow to choose different reporters (authored by Gehel).
clustershell: allow to choose different reporters
Tue, Apr 27, 5:50 PM

Mon, Apr 26

Volans added a comment to T221388: Test dhcp-option 82.

Thanks for the confirmation and sorry about the trouble @jbond

Mon, Apr 26, 3:42 PM · Patch-For-Review, SRE, netops
Volans committed rCUMINdbcfe3e14e4f: tests: fix minimum dependency and pytest warning (authored by Volans).
tests: fix minimum dependency and pytest warning
Mon, Apr 26, 12:57 PM
Volans committed rCUMIN8da80815a8f8: tests: fix integration tests (authored by Volans).
tests: fix integration tests
Mon, Apr 26, 11:11 AM
Volans added a comment to T280622: Determine safe concurrent puppet run batches via cumin.

How long did the run take? My reading of the graphs says ~40m (from 12:00 to 12:38), is that correct?

This sounds about right to me but unfortunately i don't have anything more precise still in my terminal

Mon, Apr 26, 10:35 AM · SRE, Puppet
Volans added a comment to T278137: Migrate eventlog1002 to buster.

What is the current status of eventlog1003?
It's reported by a cumin check that ensures that all hosts matching the alias A:all are part of one of the datacenters, and eventlog1003 is not part of the alias for A:eqiad.
AFAICT it's in PuppetDB but not assigned to any role in site.pp. See also https://puppetboard.wikimedia.org/node/eventlog1003.eqiad.wmnet

Mon, Apr 26, 10:10 AM · Analytics-Kanban, Analytics-Clusters

Thu, Apr 22

Volans assigned T280892: debmonitor-client.postinst: line 7: systemd-sysusers: command not found on stretch docker images to jbond.

Which version was trying to install?
The latest version of debmonitor-client does this very check, see https://gerrit.wikimedia.org/r/c/operations/software/debmonitor/+/681319

Thu, Apr 22, 7:41 AM · SRE, SRE-tools

Wed, Apr 21

Volans committed rCUMIN913a3d42a187: setup.py: revert tqdm upper limit constraint (authored by Volans).
setup.py: revert tqdm upper limit constraint
Wed, Apr 21, 3:24 PM
Volans added a comment to T280759: Site: 2 VMs for failoid.

+1, LGTM

Wed, Apr 21, 9:52 AM · vm-requests, SRE

Tue, Apr 20

Volans added a comment to T279457: Multiple host down alerts from rack C2.

@ayounsi do we have already a plan for how to manage the swap in Netbox? Should we discuss it?

Tue, Apr 20, 4:07 PM · netops, SRE, ops-codfw
Volans updated the task description for T279457: Multiple host down alerts from rack C2.
Tue, Apr 20, 4:06 PM · netops, SRE, ops-codfw
Volans committed rOSHO79ab0462ed1d: doc: fix documentation generation (authored by Volans).
doc: fix documentation generation
Tue, Apr 20, 2:16 PM
Volans committed rOSHOf84c881c4949: CHANGELOG: add changelogs for release v0.2.7 (authored by Volans).
CHANGELOG: add changelogs for release v0.2.7
Tue, Apr 20, 1:55 PM
Volans updated subscribers of T280622: Determine safe concurrent puppet run batches via cumin.

Adding @jbond that might have some insights about it.
My 2 cents are that historically we've used 15 as a safe batch number that should not cause issues (see for example https://wikitech.wikimedia.org/wiki/Cumin#Run_Puppet_only_if_last_run_failed ). But that was a while ago and I hope we can now revisit that number and maybe bump it a bit.
That said some puppet catalogs are bigger than others (like the Icinga server) and so we should use a conservative enough number that doesn't overload the puppet servers even if used with a more demanding cluster from the puppet catalog compilation point of view.

Tue, Apr 20, 8:17 AM · SRE, Puppet

Mon, Apr 19

Volans assigned T280484: debmonitor-client.service stays in failed state in case of server errors to jbond.

Assigning to @jbond that was working on this.

Mon, Apr 19, 9:12 AM · SRE-tools, SRE

Thu, Apr 15

Volans added a comment to T280251: Upgrade mysql on db1107 (m2 db master).

Anytime is ok for debmonitor.

Thu, Apr 15, 1:26 PM · SRE-tools, Recommendation-API, Performance-Team, Znuny, DBA
Volans added a comment to T251416: PXE Boot defaults to automatically reimaging (normally destroying os and all filesystemdata) on all servers.

@Marostegui yes, I didn't mention all the other efforts because a bit off topic, but firmware upgrade is too in scope and in our roadmap. And totally makes sense once we have that to integrate it into the reimage workflow so that we naturally upgrade them along with OS upgrades.

Thu, Apr 15, 8:28 AM · SRE-tools, Sustainability (Incident Followup), DC-Ops, SRE

Wed, Apr 14

Volans added a comment to T251416: PXE Boot defaults to automatically reimaging (normally destroying os and all filesystemdata) on all servers.

@LSobanski I'll try to give you some context from the SRE I/F team side of things. Any feedback will be greatly appreciated, also to help set the team's priority around those items.

Wed, Apr 14, 5:33 PM · SRE-tools, Sustainability (Incident Followup), DC-Ops, SRE

Apr 13 2021

Volans added a comment to T269272: Sign-in links from Grafana dashboards don't work when not signed into SSO.

From some quick tests it seems that the redirect works fine. Just as a note, possibly intended, if I'm logged in and open grafana.w.o it remains there "logged out".

Apr 13 2021, 3:28 PM · Patch-For-Review, User-fgiunchedi, Performance-Team (Radar), CAS-SSO, observability, SRE

Apr 1 2021

Volans added a comment to T269272: Sign-in links from Grafana dashboards don't work when not signed into SSO.

Current situation for me with what I think is a valid SSO session (if I go to idp.wikimedia.org it redirects me to https://idp.wikimedia.org/login and says Log In Successful without me entering any credential).

Apr 1 2021, 9:26 AM · Patch-For-Review, User-fgiunchedi, Performance-Team (Radar), CAS-SSO, observability, SRE
Volans added a comment to T271140: Some Data Persistence DB clusters apparently do not support IPv6.

AFAICT at least on ms-be hosts I don't see the swift processes listen on v6, shouldn't that be addressed too?

Apr 1 2021, 8:06 AM · IPv6, DBA, SRE-tools

Mar 31 2021

Volans updated the task description for T278938: Netbox IPv6 for some cloud hosts is missing or not set as primary.
Mar 31 2021, 11:16 AM · netbox, cloud-services-team (Kanban)
Volans triaged T278938: Netbox IPv6 for some cloud hosts is missing or not set as primary as High priority.
Mar 31 2021, 11:16 AM · netbox, cloud-services-team (Kanban)
Volans added a comment to T278936: Netbox: import from PuppetDB script creates VIP also if exists.

Once fixed it can be tested on gerrit1001 that has the same issue (the IPv6 not marked as primary_ip6).

Mar 31 2021, 11:01 AM · netbox
Volans added a comment to T278936: Netbox: import from PuppetDB script creates VIP also if exists.

It also updated the existing VIP with the DNS entry and changed the assigned object type:
https://netbox.wikimedia.org/extras/changelog/53663/

Mar 31 2021, 10:59 AM · netbox
Volans triaged T278936: Netbox: import from PuppetDB script creates VIP also if exists as High priority.
Mar 31 2021, 10:57 AM · netbox

Mar 29 2021

Volans committed rOHPUd488147546b0: tests: link documentation on failure (authored by Volans).
tests: link documentation on failure
Mar 29 2021, 5:06 PM
Volans claimed T278523: Decommisioning a VM failed with a key error when generating DNS.
Mar 29 2021, 4:41 PM · SRE-tools, Spicerack
Volans added a comment to T278523: Decommisioning a VM failed with a key error when generating DNS.

I think is another issue with Netbox APIs cache. Basically we get all the data at the start of the cookbook and then merge them in Python to speed up the operations. I can add a try/except there and try to gather the data again on failure.

Mar 29 2021, 4:41 PM · SRE-tools, Spicerack
Volans committed rOHPU5f8e57c2bf90: docs: convert generation script to Python (authored by Volans).
docs: convert generation script to Python
Mar 29 2021, 8:10 AM

Mar 25 2021

Volans added a comment to T265904: Remove SLAAC IPs from Ganeti hosts.

@Volans, @crusnov, what do you think about changing the status of those IPs from active to SLAAC?

Mar 25 2021, 2:28 PM · Patch-For-Review, Traffic, SRE

Mar 24 2021

Volans added a comment to T277740: Support downtiming services in our cookbooks.

@JMeybohm with the above patch, once merged and deployed, you'll be able to use icinga_hosts(["foo.bar.baz", ...], verbatim_hosts=True) to get an IcingaHosts instance that will not mangle the hostnames you'll be giving, that at that point must be valid Icinga host definitions, but not forcely hostnames.

Mar 24 2021, 2:23 PM · Prod-Kubernetes, SRE, SRE-tools
Volans added a comment to T278315: global http_proxy setting.

Let me play the devil's advocate here, I hope to not have misunderstood your intentions, correct me if I'm wrong.

Mar 24 2021, 1:40 PM · SRE, Puppet, User-jbond
Volans closed T278286: 'Failed to import' error shown by cookbook as Resolved.
Mar 24 2021, 1:12 PM · SRE-tools
Volans moved T278286: 'Failed to import' error shown by cookbook from Backlog to In Code Review on the SRE-tools board.
Mar 24 2021, 10:17 AM · SRE-tools
Volans triaged T278286: 'Failed to import' error shown by cookbook as Medium priority.
Mar 24 2021, 10:17 AM · SRE-tools
Volans claimed T278286: 'Failed to import' error shown by cookbook.
Mar 24 2021, 8:56 AM · SRE-tools

Mar 23 2021

Volans claimed T277740: Support downtiming services in our cookbooks.
Mar 23 2021, 8:56 PM · Prod-Kubernetes, SRE, SRE-tools
Volans triaged T278221: Netbox: many servers don't have Platform set as Medium priority.
Mar 23 2021, 10:30 AM · DC-Ops, netbox

Mar 22 2021

Volans committed rOSHO412871c60573: tests: fix pip backtracking (authored by Volans).
tests: fix pip backtracking
Mar 22 2021, 10:01 PM
Volans triaged T278187: Private puppet commit hook checks current state of folder, not what is staged as Medium priority.
Mar 22 2021, 9:54 PM · Puppet, SRE-tools, SRE
Volans added a comment to T278078: Request for access to mailman3-roots role for Ladsgroup.

While the ownership of mailmain within SRE is being discussed, I've raised this request in today's SRE meeting and got approved so that @Ladsgroup is unblocked.

Mar 22 2021, 4:38 PM · SRE, SRE-Access-Requests
Volans moved T278078: Request for access to mailman3-roots role for Ladsgroup from Manager/NDA Approval/Confirmation to Ready To Go on the SRE-Access-Requests board.
Mar 22 2021, 4:35 PM · SRE, SRE-Access-Requests
Volans committed rCUMINa49b3db7a74d: tests: fix pip backtracking (authored by Legoktm).
tests: fix pip backtracking
Mar 22 2021, 11:31 AM
Volans triaged T278078: Request for access to mailman3-roots role for Ladsgroup as Medium priority.

I would guess that the addition of ladsgroup to the mailman3-roots group was implicitly approved too during the last SRE meeting that approved the creation of the group, but for audit log purposes is better to have the approval here on task too.
Given that the group doesn't have an explicit approval list of people, I'm asking for @mark or @faidon to approve the request here on task.

Mar 22 2021, 10:01 AM · SRE, SRE-Access-Requests

Mar 19 2021

Volans closed T277602: Requesting access to sites from Google Search Console as Resolved.

@CGlenn I've added you to the mobile domain too am.m.wikipedia.org, I consider the approval for the whole "language". Resolving, feel free to reopen if there is any issue.

Mar 19 2021, 10:41 AM · SRE, SRE-Access-Requests
Volans closed T277804: Grant Access to wmf for TsepoThoabala as Resolved.

Patch merged, added user to the wmf group.

Mar 19 2021, 9:15 AM · SRE, Gerrit-Privilege-Requests, LDAP-Access-Requests

Mar 18 2021

Volans added a comment to T277740: Support downtiming services in our cookbooks.

Doh, I think we have naming clash here :)

Mar 18 2021, 4:14 PM · Prod-Kubernetes, SRE, SRE-tools
Volans placed T277692: Requesting access to analytics-privatedata-users for Cory Massaro up for grabs.

@Ottomata is there anything to be done on the analytics side to sync the user for the intended usage?

Mar 18 2021, 3:56 PM · SRE, SRE-Access-Requests
Volans added a comment to T277692: Requesting access to analytics-privatedata-users for Cory Massaro.

And feel free to resolve this task once it's all working as expected.

Mar 18 2021, 3:52 PM · SRE, SRE-Access-Requests
Volans added a comment to T277692: Requesting access to analytics-privatedata-users for Cory Massaro.

@cmassaro kerberos activated and patch with your access merged, please follow https://wikitech.wikimedia.org/wiki/Production_access#Setting_up_your_access to setup your SSH configuration file and test your access to one bastion first and then one of the hosts you want to access. FYI it can take up to 30 minutes from the merge time to get the change propagated to all bastions to allow you to SSH.

Mar 18 2021, 3:50 PM · SRE, SRE-Access-Requests
Volans added a comment to T277692: Requesting access to analytics-privatedata-users for Cory Massaro.

Created kerberos principal:

Mar 18 2021, 3:43 PM · SRE, SRE-Access-Requests
Volans claimed T277692: Requesting access to analytics-privatedata-users for Cory Massaro.
Mar 18 2021, 3:08 PM · SRE, SRE-Access-Requests