Page MenuHomePhabricator

crusnov (Cas Rusnov)
User

Projects

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Oct 15 2018, 5:56 PM (75 w, 6 d)
Availability
Available
LDAP User
CRusnov
MediaWiki User
CRusnov (WMF) [ Global Accounts ]

Recent Activity

Wed, Mar 18

crusnov closed T244291: Upgrade Netbox to 2.7 series as Resolved.
Wed, Mar 18, 12:05 AM · netbox, User-crusnov, SRE-tools

Fri, Mar 13

crusnov committed rOSNBd4a3a5e6e299: switch swagger to nonpublic mode (authored by crusnov).
switch swagger to nonpublic mode
Fri, Mar 13, 4:45 PM
crusnov committed rOSNB1cd3f3ca8f66: Fix imports for settings mod. (authored by crusnov).
Fix imports for settings mod.
Fri, Mar 13, 4:45 PM
crusnov committed rOSNBcec818b4dc3a: Add a passthrough configuration system (authored by crusnov).
Add a passthrough configuration system
Fri, Mar 13, 4:45 PM
crusnov added a comment to T243362: Integrate automated DNS snippets into CI.

Summary of status:

Fri, Mar 13, 4:33 PM · Release-Engineering-Team (CI & Testing services), Continuous-Integration-Config, User-jbond, Traffic, Operations, User-crusnov, Goal, SRE-tools
crusnov updated the task description for T243362: Integrate automated DNS snippets into CI.
Fri, Mar 13, 4:29 PM · Release-Engineering-Team (CI & Testing services), Continuous-Integration-Config, User-jbond, Traffic, Operations, User-crusnov, Goal, SRE-tools
crusnov triaged T216647: hiera_lookup: Allow query against checkout of labs/private in addition to checkout of operations/puppet as Lowest priority.
Fri, Mar 13, 3:44 PM · User-jbond, Puppet
crusnov added a comment to T216647: hiera_lookup: Allow query against checkout of labs/private in addition to checkout of operations/puppet.

Ah so the use case is trying to figure out the visibility-cone of hiera keys from the utils/hiera_lookup. We always place false values in the labs/private repo for private stuff, so it'd be neat if hier_lookup could look in there like CI does.

Fri, Mar 13, 3:44 PM · User-jbond, Puppet

Thu, Mar 12

crusnov committed rOSNE687e2b7edeb6: import_mgmt_dns: Update frack mgmt prefix length for codfw (authored by crusnov).
import_mgmt_dns: Update frack mgmt prefix length for codfw
Thu, Mar 12, 3:38 PM

Tue, Mar 10

crusnov triaged T247364: Forward port Python2 files to Python3 in Puppet Repository as Medium priority.
Tue, Mar 10, 10:48 PM · User-jbond, Python3-Porting, SRE-tools, Puppet
crusnov added a comment to T247364: Forward port Python2 files to Python3 in Puppet Repository.

A quick survey:

Tue, Mar 10, 10:35 PM · User-jbond, Python3-Porting, SRE-tools, Puppet
crusnov updated the task description for T247364: Forward port Python2 files to Python3 in Puppet Repository.
Tue, Mar 10, 10:33 PM · User-jbond, Python3-Porting, SRE-tools, Puppet
crusnov created T247364: Forward port Python2 files to Python3 in Puppet Repository.
Tue, Mar 10, 10:32 PM · User-jbond, Python3-Porting, SRE-tools, Puppet
crusnov committed rOSNE4f96778b8869: reports/coherence.py: Add test for racked devices with no position (authored by crusnov).
reports/coherence.py: Add test for racked devices with no position
Tue, Mar 10, 4:07 PM
crusnov committed rOSNEa65291918a02: reports/coherence.py: Add check for Juniper inventory item descriptions (authored by crusnov).
reports/coherence.py: Add check for Juniper inventory item descriptions
Tue, Mar 10, 4:07 PM
crusnov added a comment to T236582: "automation-framework" Cloud VPS project jessie deprecation.

The 2020-04-25 EOL date for Debian Jessie is rapidly approaching. Projects where the status of migration is unknown maybe subject to forced instance shutdown on or before that date. Please reply with a rough timeline/plan for the resolution of this task.

Tue, Mar 10, 3:43 PM · Cloud-VPS (Debian Jessie Deprecation)

Wed, Mar 4

crusnov added a comment to T239244: Netbox report check for no position set in rack.

In testing it seems as though the 0U height successfully prevents false negatives, but is there something additional you're trying to test for by specifying that it should only be PDUs? I think that they may be the only ones with 0 height devices.

Wed, Mar 4, 1:25 AM · Patch-For-Review, netbox, Operations

Tue, Mar 3

RobH awarded T239244: Netbox report check for no position set in rack a Like token.
Tue, Mar 3, 5:22 PM · Patch-For-Review, netbox, Operations
crusnov added a comment to T241289: Netbox report check for inventory items purchase date/task.

Do you have a particular preference as to whether this should be a new report (inventory.Purchasedate or whatever) or part of the Coherence report?

Tue, Mar 3, 5:16 PM · SRE-tools, netbox

Feb 28 2020

crusnov committed rOSNB0d5f7a6897ff: Fix imports for settings mod. (authored by crusnov).
Fix imports for settings mod.
Feb 28 2020, 6:21 PM
crusnov committed rOSNBda62ffa494f4: switch swagger to nonpublic mode (authored by crusnov).
switch swagger to nonpublic mode
Feb 28 2020, 6:21 PM
crusnov committed rOSNB0ffed69aef7c: Add a passthrough configuration system (authored by crusnov).
Add a passthrough configuration system
Feb 28 2020, 6:21 PM

Feb 27 2020

crusnov added a comment to T244849: Add SSO support to netbox.

Some notes from conversations about this:

Feb 27 2020, 9:29 PM · netbox, Operations

Feb 25 2020

crusnov closed T238919: Cleanup Netbox stuff from netmon hosts as Resolved.

I have done a survey of the hosts and I believe these nits are cleared up and cleaned.

Feb 25 2020, 8:11 PM · netbox
crusnov added a project to T244153: Import PuppetDB interfaces into Netbox: DC-Ops.

So there's a decision point that is blocking this project.

Feb 25 2020, 8:06 PM · DC-Ops, SRE-tools, netbox, User-crusnov
crusnov added a project to T229397: Puppet: get row/rack info from Netbox: User-crusnov.
Feb 25 2020, 5:07 PM · User-crusnov, User-jbond, Patch-For-Review, Puppet, Operations
crusnov moved T222931: Netbox Reports Ideas and Requests from Ready to Living / Watching on the User-crusnov board.
Feb 25 2020, 5:04 PM · netbox, User-crusnov, SRE-tools
crusnov edited P10499 netbox premetheus device stats.
Feb 25 2020, 4:30 PM

Feb 24 2020

crusnov moved T239119: Netbox reports can break Netbox sometimes from Pending to Complete on the User-crusnov board.
Feb 24 2020, 11:33 PM · User-crusnov, SRE-tools, netbox
crusnov closed T239119: Netbox reports can break Netbox sometimes as Resolved.

I have merged these changes. This bug is complete.

Feb 24 2020, 11:33 PM · User-crusnov, SRE-tools, netbox
crusnov created P10499 netbox premetheus device stats.
Feb 24 2020, 4:19 PM

Feb 21 2020

crusnov moved T244291: Upgrade Netbox to 2.7 series from In Progress to Complete on the User-crusnov board.
Feb 21 2020, 7:41 PM · netbox, User-crusnov, SRE-tools
crusnov moved T239119: Netbox reports can break Netbox sometimes from In Progress to Pending on the User-crusnov board.
Feb 21 2020, 7:41 PM · User-crusnov, SRE-tools, netbox

Feb 20 2020

crusnov added a comment to T244849: Add SSO support to netbox.

On a practical level we already maintain a fork, so if any changes are needed they can be integrated into our fork (we should wait until the post-upgrade ~this week though).

We should not maintain a fork permanently. Backporting patches from newer releases/master, or applying patches that we've pushed upstream is fine, but permanently deviating from the upstream codebase is not.

Feb 20 2020, 8:01 PM · netbox, Operations

Feb 19 2020

crusnov updated the task description for T244291: Upgrade Netbox to 2.7 series.
Feb 19 2020, 1:23 AM · netbox, User-crusnov, SRE-tools
crusnov added a comment to T244291: Upgrade Netbox to 2.7 series.

All external scripts sometimes fail (dump and ganeti sync in particular).

Feb 19 2020, 1:23 AM · netbox, User-crusnov, SRE-tools
crusnov added a comment to T244291: Upgrade Netbox to 2.7 series.

Report issues resolved.

Feb 19 2020, 1:18 AM · netbox, User-crusnov, SRE-tools
crusnov added a comment to T244291: Upgrade Netbox to 2.7 series.

Okay fallout from upgrade:

Feb 19 2020, 12:03 AM · netbox, User-crusnov, SRE-tools

Feb 18 2020

crusnov updated the task description for T244291: Upgrade Netbox to 2.7 series.
Feb 18 2020, 10:56 PM · netbox, User-crusnov, SRE-tools
crusnov updated the task description for T244291: Upgrade Netbox to 2.7 series.
Feb 18 2020, 10:44 PM · netbox, User-crusnov, SRE-tools
crusnov updated the task description for T244291: Upgrade Netbox to 2.7 series.
Feb 18 2020, 10:37 PM · netbox, User-crusnov, SRE-tools
crusnov added a comment to T239119: Netbox reports can break Netbox sometimes.

I have completed the non-breaking changes successfully to puppetdb report. I will execute the rest now.

Feb 18 2020, 10:32 PM · User-crusnov, SRE-tools, netbox
crusnov updated the task description for T244291: Upgrade Netbox to 2.7 series.
Feb 18 2020, 7:48 PM · netbox, User-crusnov, SRE-tools
crusnov moved T244291: Upgrade Netbox to 2.7 series from Up next to In Progress on the SRE-tools board.
Feb 18 2020, 5:43 PM · netbox, User-crusnov, SRE-tools
crusnov updated the task description for T244291: Upgrade Netbox to 2.7 series.
Feb 18 2020, 5:30 PM · netbox, User-crusnov, SRE-tools

Feb 14 2020

crusnov added a comment to T244291: Upgrade Netbox to 2.7 series.

Once all of the changes are approved, the process will be thus:

Feb 14 2020, 8:24 PM · netbox, User-crusnov, SRE-tools
crusnov moved T239119: Netbox reports can break Netbox sometimes from Backlog to In Progress on the User-crusnov board.
Feb 14 2020, 8:15 PM · User-crusnov, SRE-tools, netbox
crusnov moved T240986: Make netbox-extras deploy via air-gap from Backlog to Ready on the User-crusnov board.
Feb 14 2020, 8:14 PM · SRE-tools, netbox, User-crusnov
crusnov moved T222931: Netbox Reports Ideas and Requests from Backlog to Ready on the User-crusnov board.
Feb 14 2020, 8:14 PM · netbox, User-crusnov, SRE-tools
crusnov moved T233183: Automate generation of Management DNS records from Netbox from Backlog to In Progress on the User-crusnov board.
Feb 14 2020, 8:14 PM · Patch-For-Review, User-jbond, Traffic, Operations, User-crusnov, Goal, SRE-tools
crusnov moved T244291: Upgrade Netbox to 2.7 series from Ready to In Progress on the User-crusnov board.
Feb 14 2020, 8:11 PM · netbox, User-crusnov, SRE-tools

Feb 12 2020

crusnov committed rOSNBb1b8d71b6545: switch swagger to nonpublic mode (authored by crusnov).
switch swagger to nonpublic mode
Feb 12 2020, 5:43 PM
crusnov committed rOSNB284b215d7b66: Fix imports for settings mod. (authored by crusnov).
Fix imports for settings mod.
Feb 12 2020, 5:43 PM
crusnov committed rOSNBb46b4691edf8: Add a passthrough configuration system (authored by crusnov).
Add a passthrough configuration system
Feb 12 2020, 5:43 PM

Feb 11 2020

crusnov added a comment to T244849: Add SSO support to netbox.

On a practical level we already maintain a fork, so if any changes are needed they can be integrated into our fork (we should wait until the post-upgrade ~this week though).

Feb 11 2020, 5:26 PM · netbox, Operations

Feb 4 2020

crusnov moved T244291: Upgrade Netbox to 2.7 series from Backlog to Ready on the User-crusnov board.
Feb 4 2020, 9:29 PM · netbox, User-crusnov, SRE-tools
crusnov moved T243927: Export Netbox Stats for DCops to a visualization tool from Ready to In Progress on the User-crusnov board.
Feb 4 2020, 9:28 PM · Patch-For-Review, SRE-tools, netbox, DC-Ops, User-crusnov
crusnov triaged T244291: Upgrade Netbox to 2.7 series as High priority.
Feb 4 2020, 9:26 PM · netbox, User-crusnov, SRE-tools
crusnov created T244291: Upgrade Netbox to 2.7 series.
Feb 4 2020, 9:26 PM · netbox, User-crusnov, SRE-tools
crusnov added a comment to T242705: ORES uwsgi consumes a large amount of memory and CPU when shutting down (as part of a restart).

Just a quick note, an extremely similar event has occurred today at 0630 UTC. The entire ORES cluster seems to have had oom issues for a while.

Feb 4 2020, 6:40 AM · Scoring-platform-team (Current), Operations, ORES

Feb 3 2020

crusnov moved T243362: Integrate automated DNS snippets into CI from Backlog to In Progress on the User-crusnov board.
Feb 3 2020, 8:54 PM · Release-Engineering-Team (CI & Testing services), Continuous-Integration-Config, User-jbond, Traffic, Operations, User-crusnov, Goal, SRE-tools
crusnov added a comment to T244153: Import PuppetDB interfaces into Netbox.

Importantly we decided that it would *not* import MAC since our next strategy for initial DHCP configuration will be MACless strategy of some variety.

Feb 3 2020, 6:19 PM · DC-Ops, SRE-tools, netbox, User-crusnov
crusnov moved T244153: Import PuppetDB interfaces into Netbox from Ready to In Progress on the User-crusnov board.
Feb 3 2020, 6:04 PM · DC-Ops, SRE-tools, netbox, User-crusnov
crusnov created T244153: Import PuppetDB interfaces into Netbox.
Feb 3 2020, 6:04 PM · DC-Ops, SRE-tools, netbox, User-crusnov

Jan 29 2020

crusnov added projects to T243927: Export Netbox Stats for DCops to a visualization tool: DC-Ops, netbox, SRE-tools.
Jan 29 2020, 11:11 PM · Patch-For-Review, SRE-tools, netbox, DC-Ops, User-crusnov
crusnov added a project to T243928: Complete Netbox prometheus scraping: SRE-tools.
Jan 29 2020, 11:11 PM · SRE-tools, netbox, User-crusnov
crusnov created T243928: Complete Netbox prometheus scraping.
Jan 29 2020, 11:11 PM · SRE-tools, netbox, User-crusnov
crusnov created T243927: Export Netbox Stats for DCops to a visualization tool.
Jan 29 2020, 10:55 PM · Patch-For-Review, SRE-tools, netbox, DC-Ops, User-crusnov
crusnov added a comment to T242438: CAS SSO: failed u2f registration.

Here's a separate avenue that may be related to this. So the traceback from Java shows a bad signature, and I dug into this a bit with u2f testing sites.

Jan 29 2020, 12:09 AM · User-jbond, Operations

Jan 23 2020

crusnov added a comment to T237466: Remove unused custom fields from Netbox.

I have created the backup dump pre-t237466-2020-01-23-17:37 and deleted support_until and support_contract. I shall keep an eye on it for any breakage, and do please let me know.

Jan 23 2020, 6:17 PM · SRE-tools, DC-Ops, netbox
crusnov added a comment to T237466: Remove unused custom fields from Netbox.

There is no specific reference to those two fields anywhere in the support software (reports, automated systems, etc). It does not appear we need to patch any of that to remove them.

Jan 23 2020, 5:24 PM · SRE-tools, DC-Ops, netbox
crusnov added a comment to T237466: Remove unused custom fields from Netbox.

Cool this should be pretty straight forward.

Jan 23 2020, 5:13 PM · SRE-tools, DC-Ops, netbox

Jan 22 2020

crusnov added a comment to T238350: Merge all netbox extras into one repository.

Still need to clean up old repository checkout, and make sure that Netbox is entirely using the entries from netbox-extras.

Jan 22 2020, 12:57 AM · SRE-tools, netbox
crusnov added a comment to T243362: Integrate automated DNS snippets into CI.

The pulling the git looks straight forward (adding to the Dockerfile), as for actually testing against the pull I suspect we'll have to use the template mechanics that builds the DNS repository to pass a path to the path those snippets are checked out to so that any place we $INCLUDE a snippet, we use a template variable (such as {{INCLUDEPATH}} or somesuch) - correct me if I'm wrong @BBlack but looking at the docs for gdnsd it doesn't appear that there is a way to set the search path for $INCLUDES otherwise.

Jan 22 2020, 12:36 AM · Release-Engineering-Team (CI & Testing services), Continuous-Integration-Config, User-jbond, Traffic, Operations, User-crusnov, Goal, SRE-tools
crusnov created T243362: Integrate automated DNS snippets into CI.
Jan 22 2020, 12:33 AM · Release-Engineering-Team (CI & Testing services), Continuous-Integration-Config, User-jbond, Traffic, Operations, User-crusnov, Goal, SRE-tools

Jan 16 2020

crusnov claimed T237466: Remove unused custom fields from Netbox.
Jan 16 2020, 9:57 PM · SRE-tools, DC-Ops, netbox

Jan 15 2020

crusnov added a comment to T230531: Run Matrix trial using the modular.im-hosted instance.

I feel as though hosting our own instances is preferable, overall, in production. I understand for prototyping / buy-in using a hosted service is desirable, but the actual overhead for hosting is not that great. I get that volunteering other people's time isn't something that could be done, but I can volunteer a little of my SRE time to this prototyping phase in any case.

Jan 15 2020, 5:01 PM · User-brennen, User-ArielGlenn, Matrix

Jan 9 2020

crusnov added a comment to T230531: Run Matrix trial using the modular.im-hosted instance.

For what it's worth, if any hands are needed I can lend some.

Jan 9 2020, 12:18 AM · User-brennen, User-ArielGlenn, Matrix

Jan 7 2020

crusnov created P10047 rotatebackup results.
Jan 7 2020, 5:44 AM · netbox

Jan 3 2020

crusnov committed rOSNE4c77272587aa: tools/import-mgmt-dns.py: General Improvements (authored by crusnov).
tools/import-mgmt-dns.py: General Improvements
Jan 3 2020, 8:16 PM
crusnov added a comment to T228387: Bare metal cloud: management interfaces.

Alright I have adjusted PDUs to use 100M interface types (on all but esams) and unset the mgmt_only bit.

Jan 3 2020, 5:29 PM · User-crusnov, Goal, SRE-tools
crusnov added a comment to T228387: Bare metal cloud: management interfaces.

Cool, I have executed the script which has imported them. I didn't change the interface type from 1G ones, so those are wrong, and tehy do have the mgmt_only bit set, so i'll just bulk fix that.

Jan 3 2020, 5:02 PM · User-crusnov, Goal, SRE-tools
crusnov added a comment to T228387: Bare metal cloud: management interfaces.

Setup script to import PDUs too, here's what the results would be:
https://phabricator.wikimedia.org/P10027

Jan 3 2020, 6:17 AM · User-crusnov, Goal, SRE-tools
crusnov added a comment to T228387: Bare metal cloud: management interfaces.

Imported Decommissioning and Failed status DNS entries. See paste at: https://phabricator.wikimedia.org/P10026 for list.

Jan 3 2020, 4:28 AM · User-crusnov, Goal, SRE-tools

Dec 23 2019

crusnov added a comment to T228387: Bare metal cloud: management interfaces.

After tidying, DNS has been reimported successfully.

Dec 23 2019, 7:04 PM · User-crusnov, Goal, SRE-tools
crusnov added a comment to T228387: Bare metal cloud: management interfaces.

Some conflicts in importing DNS again because of offline devices. https://phabricator.wikimedia.org/P10013 is a list.

Dec 23 2019, 6:54 PM · User-crusnov, Goal, SRE-tools
crusnov added a comment to T228387: Bare metal cloud: management interfaces.

Okay here's an updated log (I took out the No Information lines)

Dec 23 2019, 6:20 PM · User-crusnov, Goal, SRE-tools
crusnov added a comment to T228387: Bare metal cloud: management interfaces.

For what it's worth the latest import of up-to-date dns repository, with the current state of things (dry-run):

Dec 23 2019, 7:05 AM · User-crusnov, Goal, SRE-tools
crusnov added a comment to T239123: Netbox: Synchronize ganeti VMs from new clusters.

esams and ulsfo added to sync. Leaving this open to track for when the eqsin cluster is available.

Dec 23 2019, 5:32 AM · User-crusnov, SRE-tools, netbox

Dec 21 2019

crusnov merged task T241166: Sync new ganeti clusters with netbox into T239123: Netbox: Synchronize ganeti VMs from new clusters.
Dec 21 2019, 12:38 AM · Operations, netbox
crusnov merged T241166: Sync new ganeti clusters with netbox into T239123: Netbox: Synchronize ganeti VMs from new clusters.
Dec 21 2019, 12:38 AM · User-crusnov, SRE-tools, netbox
crusnov added a comment to T241166: Sync new ganeti clusters with netbox.

Yes this is a duplicate. I'll merge it.

Dec 21 2019, 12:37 AM · Operations, netbox

Dec 20 2019

crusnov created T241259: Unify netbox tokens in private repo.
Dec 20 2019, 5:30 PM · User-crusnov

Dec 18 2019

crusnov triaged T241074: Upgrade pynetbox package to 4.2.2 as Medium priority.
Dec 18 2019, 4:19 PM · SRE-tools, netbox, User-crusnov
crusnov created T241074: Upgrade pynetbox package to 4.2.2.
Dec 18 2019, 4:19 PM · SRE-tools, netbox, User-crusnov
crusnov committed rOSNEaa781b9bd191: Import various tools from netbox-deploy as part of unification (authored by crusnov).
Import various tools from netbox-deploy as part of unification
Dec 18 2019, 3:39 AM

Dec 17 2019

crusnov triaged T240986: Make netbox-extras deploy via air-gap as Medium priority.
Dec 17 2019, 7:22 PM · SRE-tools, netbox, User-crusnov
crusnov created T240986: Make netbox-extras deploy via air-gap.
Dec 17 2019, 7:21 PM · SRE-tools, netbox, User-crusnov

Dec 16 2019

crusnov added a comment to T240176: OKR: Provide a mechanism to obtain the FQDN given a short hostname.

We shall import this information from PuppetDB since this also contains the network device information which is currently missing from Netbox. Of course this does not cover non-host devices, but gets us mostly there in terms of primary IP addresses, hostnames, device names, etc.

Dec 16 2019, 5:36 PM · netbox, SRE-tools
crusnov added a comment to T238350: Merge all netbox extras into one repository.

I have merged the switchover to extras, tested the reports and customscripts, and mv the exsiting checkout of netbox-reports out of the way to make sure netbox is using the new repo. Everything seems to be working correctly, we are now using this repository for reports, customscripts and extra tools for Netbox.

Dec 16 2019, 5:11 PM · SRE-tools, netbox