Page MenuHomePhabricator

crusnov (Cas Rusnov)
User

Projects

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Oct 15 2018, 5:56 PM (60 w, 5 d)
Availability
Available
LDAP User
CRusnov
MediaWiki User
Unknown

Recent Activity

Mon, Dec 9

crusnov added a comment to T240176: OKR: Provide a mechanism to obtain the FQDN given a short hostname.

I have been working on adapting the current DNS import script to additionally import other DNS entries and create primary IP addresses on hosts in Netbox. This should provide sufficient information in most cases to obtain the FQDN with an extremely simple one liner if the API is already available.

Mon, Dec 9, 6:31 AM · netbox, SRE-tools
crusnov triaged T240176: OKR: Provide a mechanism to obtain the FQDN given a short hostname as Medium priority.
Mon, Dec 9, 6:30 AM · netbox, SRE-tools
crusnov created T240176: OKR: Provide a mechanism to obtain the FQDN given a short hostname.
Mon, Dec 9, 6:30 AM · netbox, SRE-tools
crusnov lowered the priority of T231068: Spicerack: improve support for Ganeti VMs from Medium to Low.
Mon, Dec 9, 6:28 AM · SRE-tools
crusnov added a subtask for T223292: Netbox: generate CSV backups: T231512: Netbox: Add CSV dump rotation.
Mon, Dec 9, 6:26 AM · netbox
crusnov added a parent task for T231512: Netbox: Add CSV dump rotation: T223292: Netbox: generate CSV backups.
Mon, Dec 9, 6:26 AM · SRE-tools
crusnov added a comment to T239119: Netbox reports can break Netbox sometimes.

I will complete this as soon as we complete migrating to the netbox-extras repositor.

Mon, Dec 9, 6:25 AM · User-crusnov, SRE-tools, netbox
crusnov added a comment to T231512: Netbox: Add CSV dump rotation.

I will circle back to the discussed changes this week.

Mon, Dec 9, 6:25 AM · SRE-tools
crusnov closed T237469: Netbox: Fix hostname case ambiguity as Resolved.

afaict this is complete. The patch has been merged which performs the check alluded to in the op.

Mon, Dec 9, 6:24 AM · netbox, DC-Ops

Thu, Dec 5

crusnov committed rOSNE8f4772fa4c28: Update README.md to reflect the use of this repository (authored by crusnov).
Update README.md to reflect the use of this repository
Thu, Dec 5, 11:20 PM
crusnov committed rOSNEe3f0baee0c2f: Merge reports and customscripts from netbox-reports (authored by crusnov).
Merge reports and customscripts from netbox-reports
Thu, Dec 5, 11:20 PM
crusnov committed rOSNEf7a88666d851: remove reports sepicific tox.ini (authored by crusnov).
remove reports sepicific tox.ini
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE25d00f84f171: coherence: Check device names for correct case (authored by crusnov).
coherence: Check device names for correct case
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE9ec2eba5d2df: cables: detect duplicate cable names, and blank cable names (authored by crusnov).
cables: detect duplicate cable names, and blank cable names
Thu, Dec 5, 11:20 PM
crusnov committed rOSNEe2c219b37797: cables: fix sense of blankness check (authored by crusnov).
cables: fix sense of blankness check
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE450e22c71e55: cables: blacklist eqiad in blank cable test (authored by crusnov).
cables: blacklist eqiad in blank cable test
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE70a6dfbf8646: coherence: Check device names for correct formatting (authored by crusnov).
coherence: Check device names for correct formatting
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE6af11e2df22b: librenms: Handle the case where hardware is null (authored by crusnov).
librenms: Handle the case where hardware is null
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE65a4d968673c: coherence: Check unracked devices for connected console ports (authored by crusnov).
coherence: Check unracked devices for connected console ports
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE96ca950e0d1a: interface_automation: fix refactor error (authored by crusnov).
interface_automation: fix refactor error
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE14c450986fd2: interface_automation: fix minor issues (authored by crusnov).
interface_automation: fix minor issues
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE6b3bea1d3b37: librenms: Exclude problematic InventoryItem type as requested (authored by crusnov).
librenms: Exclude problematic InventoryItem type as requested
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE9834d75096ff: Initial support for custom scripts (authored by crusnov).
Initial support for custom scripts
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE8b68951bcf9e: librenms: Fix for api shift (authored by crusnov).
librenms: Fix for api shift
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE4b8ac38a1889: Add LibreNMS parity check report (authored by crusnov).
Add LibreNMS parity check report
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE7129f5811065: Add cable names report (authored by crusnov).
Add cable names report
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE3787e78f9741: Move netbox report config to /etc/netbox (authored by crusnov).
Move netbox report config to /etc/netbox
Thu, Dec 5, 11:20 PM
crusnov committed rOSNEf967c7656734: Revert "Also exclude `FAILED` state from PuppetDB reports" (authored by crusnov).
Revert "Also exclude `FAILED` state from PuppetDB reports"
Thu, Dec 5, 11:20 PM
crusnov committed rOSNEcfd27e6e14ca: Also exclude `FAILED` state from PuppetDB reports (authored by crusnov).
Also exclude `FAILED` state from PuppetDB reports
Thu, Dec 5, 11:20 PM
crusnov added a reverting change for rOSNEcfd27e6e14ca: Also exclude `FAILED` state from PuppetDB reports: rOSNEf967c7656734: Revert "Also exclude `FAILED` state from PuppetDB reports".
Thu, Dec 5, 11:20 PM
crusnov committed rOSNEb91bc516fcf7: Fix minor typo in oldhardware report (authored by crusnov).
Fix minor typo in oldhardware report
Thu, Dec 5, 11:20 PM
crusnov committed rOSNEf00c37d1fe05: fix minor typo in puppetdb report (authored by crusnov).
fix minor typo in puppetdb report
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE2c6f49262099: Cleanups to the oldhardware report (authored by crusnov).
Cleanups to the oldhardware report
Thu, Dec 5, 11:20 PM
crusnov committed rOSNE24bc7834a9a5: Minor improvements to management console report (authored by crusnov).
Minor improvements to management console report
Thu, Dec 5, 11:19 PM
crusnov committed rOSNEe4de619b37ac: Add device model/device type parity check (authored by crusnov).
Add device model/device type parity check
Thu, Dec 5, 11:19 PM
crusnov committed rOSNEf2e1dbbae56d: Fix minor error in date compare (authored by crusnov).
Fix minor error in date compare
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE84cc7781b12c: Minor improvements to PuppetDB report (authored by crusnov).
Minor improvements to PuppetDB report
Thu, Dec 5, 11:19 PM
crusnov committed rOSNEc3ec85cf6f2c: Yet more Coherence fixes (authored by crusnov).
Yet more Coherence fixes
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE2cdd17a98bcb: puppetdb report: Exclude OFFLINE VMs from report (authored by crusnov).
puppetdb report: Exclude OFFLINE VMs from report
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE5e00af184345: coherence report: General improvements and rack checks (authored by crusnov).
coherence report: General improvements and rack checks
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE971aa69ae490: Add README.md and LICENSE.txt (authored by crusnov).
Add README.md and LICENSE.txt
Thu, Dec 5, 11:19 PM
crusnov committed rOSNEd8aefdcb3a82: Change black check to not enforce quote style. (authored by crusnov).
Change black check to not enforce quote style.
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE5813774f3812: Minor fixes (authored by crusnov).
Minor fixes
Thu, Dec 5, 11:19 PM
crusnov committed rOSNEbda6d17ebd38: Break report into parts and adjust the way devices are filtered (authored by crusnov).
Break report into parts and adjust the way devices are filtered
Thu, Dec 5, 11:19 PM
crusnov committed rOSNEc1080b883bc8: Make oldhardware report exclued cablemgmt and storagebin (authored by crusnov).
Make oldhardware report exclued cablemgmt and storagebin
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE1b2fe3f80836: Reorganize and add tox/CI support for repository. (authored by crusnov).
Reorganize and add tox/CI support for repository.
Thu, Dec 5, 11:19 PM
crusnov committed rOSNEad28fb6da593: Add report which checks against puppetdb and compares serial numbers (authored by crusnov).
Add report which checks against puppetdb and compares serial numbers
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE8434ca18ef5f: management.py: trivial change to adapt for Netbox 2.5+ (authored by crusnov).
management.py: trivial change to adapt for Netbox 2.5+
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE7af2ca8dfd72: Properly detect connected ports. (authored by crusnov).
Properly detect connected ports.
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE435f1c82979a: Change all reports to log only errors except for a summary count (authored by crusnov).
Change all reports to log only errors except for a summary count
Thu, Dec 5, 11:19 PM
crusnov committed rOSNEa15df86f3abd: Add management console report (authored by crusnov).
Add management console report
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE03e7a0136c3e: Fix typo in oldhardware report. (authored by crusnov).
Fix typo in oldhardware report.
Thu, Dec 5, 11:19 PM
crusnov committed rOSNE073520649269: Add an old hardware report (authored by crusnov).
Add an old hardware report
Thu, Dec 5, 11:19 PM
crusnov committed rOSNEcc3df8ffbb12: Add "Coherence" check (authored by crusnov).
Add "Coherence" check
Thu, Dec 5, 11:19 PM

Wed, Dec 4

crusnov added a comment to T236582: "automation-framework" Cloud VPS project jessie deprecation.

Hello, we will take a look at that. It is possible for this to get done by the deadline, but there are some minor unknowns due to local changes in our puppetmaster. Thanks for the ping.

Wed, Dec 4, 6:54 PM · Cloud-VPS (Debian Jessie Deprecation)

Tue, Nov 26

crusnov added a comment to T237464: Netbox Coherence Report enhancements .

I think the general consensus is "spare" or "future" anywhere in the hostname is an error if the host is ACTIVE in netbox.

Tue, Nov 26, 4:59 PM · Patch-For-Review, netbox
crusnov added a comment to T233183: Automate generation of Management DNS records from Netbox.

After a conversation with @Volans an extended ask is having the generator able to add and remove files (eg, override completely the contents of repository, if necessary). This is a bit of an extension of the workflow I'd envisioned but I shall be implementing that now.

Tue, Nov 26, 4:58 PM · User-jbond, Traffic, Operations, Patch-For-Review, User-crusnov, Goal, SRE-tools
crusnov added a comment to T239123: Netbox: Synchronize ganeti VMs from new clusters.

I've gone ahead and added the clusters in Netbox.

Tue, Nov 26, 4:55 PM · User-crusnov, SRE-tools, netbox
crusnov updated the task description for T239123: Netbox: Synchronize ganeti VMs from new clusters.
Tue, Nov 26, 4:55 PM · User-crusnov, SRE-tools, netbox
crusnov added a comment to T238919: Cleanup Netbox stuff from netmon hosts.

@Volans what do you mean by "any remaining puppet code" ?

Tue, Nov 26, 4:52 PM · netbox
crusnov added a comment to T238919: Cleanup Netbox stuff from netmon hosts.

And rechecking the top list, I have removed acme certs, and other miscellany from /etc.

Tue, Nov 26, 4:49 PM · netbox
crusnov added a comment to T238919: Cleanup Netbox stuff from netmon hosts.

Okay I believe I have removed all traces on netmo1002 and netmon2001:

Tue, Nov 26, 4:46 PM · netbox

Mon, Nov 25

crusnov added a comment to T237469: Netbox: Fix hostname case ambiguity.

As discussed on IRC, the above solution is agreed upon. This should not cause any false positives.

Mon, Nov 25, 6:52 PM · netbox, DC-Ops
crusnov added a comment to T238919: Cleanup Netbox stuff from netmon hosts.

Thanks, I'll go ahead and remove it then.

Mon, Nov 25, 4:51 PM · netbox
crusnov triaged T239123: Netbox: Synchronize ganeti VMs from new clusters as Medium priority.
Mon, Nov 25, 4:47 PM · User-crusnov, SRE-tools, netbox
crusnov created T239123: Netbox: Synchronize ganeti VMs from new clusters.
Mon, Nov 25, 4:47 PM · User-crusnov, SRE-tools, netbox
crusnov lowered the priority of T219908: Build an API for generating boot options for iPXE from Netbox et al. based on Serial Number from Medium to Low.
Mon, Nov 25, 4:44 PM · User-crusnov, SRE-tools
crusnov added a comment to T231068: Spicerack: improve support for Ganeti VMs.

The non-optional ask on this is complete. I will leave this open to track the gnt* command proxying.

Mon, Nov 25, 4:42 PM · SRE-tools
crusnov closed T232767: Netbox API Occasionally 500s and Netbox2001 dumpcsv fails as Resolved.

This is resolved.

Mon, Nov 25, 4:42 PM · SRE-tools
crusnov closed T233728: Netbox: netbox_dump_run service failed, a subtask of T232767: Netbox API Occasionally 500s and Netbox2001 dumpcsv fails, as Resolved.
Mon, Nov 25, 4:40 PM · SRE-tools
crusnov closed T233728: Netbox: netbox_dump_run service failed as Resolved.

This appears to be resolved.

Mon, Nov 25, 4:40 PM · netbox
crusnov added a comment to T235550: Rename multimedia-team to structured-data-team.

Passing to next clinic duty person.

Mon, Nov 25, 4:40 PM · Patch-For-Review, Wikimedia-Mailing-lists, Operations
crusnov placed T235550: Rename multimedia-team to structured-data-team up for grabs.
Mon, Nov 25, 4:39 PM · Patch-For-Review, Wikimedia-Mailing-lists, Operations
crusnov added a comment to T238919: Cleanup Netbox stuff from netmon hosts.

Doing a quick check on netmon2001:

Mon, Nov 25, 4:39 PM · netbox
crusnov added a comment to T223292: Netbox: generate CSV backups.

Yep, revisiting the rotation right now. We in any case have not *lost* anything, it is just non-optimal.

Mon, Nov 25, 4:34 PM · netbox
crusnov closed T234452: Puppet breakage in automation-framework VMs, a subtask of T228866: CloudVPS: VMs with broken puppet 2019-07-14, as Resolved.
Mon, Nov 25, 4:33 PM · cloud-services-team (Kanban)
crusnov closed T234452: Puppet breakage in automation-framework VMs as Resolved.

This should be resolved, I've spot checked hosts in the af project and they have been running puppet normally.

Mon, Nov 25, 4:33 PM · Operations
crusnov closed T233624: ganeti netbox sync alerts are noisy as Resolved.

This should be resolved.

Mon, Nov 25, 4:32 PM · Operations
crusnov added a comment to T237469: Netbox: Fix hostname case ambiguity.

I guess without imposing too much, the ask was "all hostnames are lowercase" - we could just check if lower(hostname) != hostname and call it good, punting the normalization to a further task.

Mon, Nov 25, 4:31 PM · netbox, DC-Ops
crusnov triaged T239119: Netbox reports can break Netbox sometimes as High priority.
Mon, Nov 25, 4:09 PM · User-crusnov, SRE-tools, netbox
crusnov created T239119: Netbox reports can break Netbox sometimes.
Mon, Nov 25, 4:09 PM · User-crusnov, SRE-tools, netbox
crusnov closed T237803: Netbox reports Icinga checks timeout as Resolved.

I executed the plan that Riccardo outlined, removed the running ability in the check and switched to running from the management script, which has simplified the code a bit, although the real causes of the timeouts were that Netbox initializes all of the report objects when you query the .all for the reports list, which for accounting, librenms, and puppetdb involve actually accessing a remote service with unpredictable amounts of time involved. I switched the icinga check to .get the report object instead, so we only eat the unpredictability of one report which for the time being appears to be under the 10 second limit. I'm opening an additional ticket to try to defensively restructure the reports so they don't actually access external services unless they are used so to reduce any possibility of this happening (and also reduce the possibility of a broken external service preventing looking at the report list in the interface).

Mon, Nov 25, 4:05 PM · Operations, SRE-tools, netbox

Fri, Nov 22

crusnov triaged T238960: Conversion to volunteer NDA for MaxSem as Medium priority.
Fri, Nov 22, 11:00 PM · LDAP-Access-Requests, Operations, SRE-Access-Requests
crusnov added a comment to T238868: Requesting access to sites from Google Search Console.

Until september 2020 seems a reasonable timeframe (the docs say "typically aronud one year").

Fri, Nov 22, 10:23 PM · Operations, SRE-Access-Requests, GLOW
crusnov triaged T238905: Allow analytics-search-users members to sudo as the airflow user as Medium priority.
Fri, Nov 22, 8:09 PM · Operations, SRE-Access-Requests, Discovery-Search
crusnov added a comment to T238868: Requesting access to sites from Google Search Console.

According to the procedure for this request, end-dates for rechecking access are needed. Do you have an end-date in mind? Otherwise we should be able to add the access.

Fri, Nov 22, 8:07 PM · Operations, SRE-Access-Requests, GLOW
crusnov triaged T238868: Requesting access to sites from Google Search Console as Medium priority.
Fri, Nov 22, 8:03 PM · Operations, SRE-Access-Requests, GLOW
crusnov closed T236321: Requesting Access to Stat1004, Stat1006, Stat1007, notebook1003 and notebook1004 as Resolved.

Hello I have added the key above to the patch and merged it. This means that shortly (within 30 or so minutes) the key should be propagated to the appropriate bastions.

Fri, Nov 22, 7:55 PM · Operations, SRE-Access-Requests
crusnov placed T238090: Search Console access for he.wikisource.org up for grabs.

Giving this to the next person on clinic duty. We still need to know the time limits and I believe some other information to complete this process.

Fri, Nov 22, 6:52 PM · Product-Analytics, Operations, SRE-Access-Requests
crusnov closed T238759: Create OpenGLAM mailing list as Resolved.

Hello! I have created the mailing list as requested.

Fri, Nov 22, 6:47 PM · Operations, Wikimedia-Mailing-lists
crusnov added a comment to T238919: Cleanup Netbox stuff from netmon hosts.

Also the configurations in /etc/netbox, anything related to deploys in /srv

Fri, Nov 22, 4:22 PM · netbox
crusnov claimed T238090: Search Console access for he.wikisource.org.
Fri, Nov 22, 12:21 AM · Product-Analytics, Operations, SRE-Access-Requests
crusnov added a comment to T238090: Search Console access for he.wikisource.org.

Hello! The procedure to complete this suggests a time limit, what is the final say on that in this case?

Fri, Nov 22, 12:18 AM · Product-Analytics, Operations, SRE-Access-Requests

Thu, Nov 21

crusnov closed T237007: Add a Netbox check for duplicate cable IDs as Resolved.

Okay the additional cable checks are in place, and appear to be correct.

Thu, Nov 21, 6:48 PM · DC-Ops, SRE-tools, netbox
crusnov triaged T238833: Create NRPE check to alert when cergen certificates are due to expire as Medium priority.
Thu, Nov 21, 4:10 PM · Patch-For-Review, User-jbond, Puppet, Operations

Wed, Nov 20

crusnov added a comment to T237803: Netbox reports Icinga checks timeout.

The above patch should address these issues. It hugely simplifies the nagios check script and also uses the API more efficiently so it shouldn't flap anymore on a failed report.

Wed, Nov 20, 11:20 PM · Operations, SRE-tools, netbox
crusnov triaged T238733: Push rights on https://gerrit.wikimedia.org/r/admin/projects/wikidata/query/blazegraph for onimisionipe as Medium priority.
Wed, Nov 20, 4:58 PM · Gerrit-Privilege-Requests, Wikidata, Release-Engineering-Team (Unit & Int & System Tooling), Operations, Wikidata-Query-Service
crusnov triaged T238724: ATS logs aren't being rotated as Medium priority.
Wed, Nov 20, 4:57 PM · Operations, Traffic
crusnov triaged T238727: Include zone+subnet checks for DNS validation as Medium priority.
Wed, Nov 20, 4:57 PM · Traffic, Operations, DNS, SRE-tools
crusnov updated subscribers of T237464: Netbox Coherence Report enhancements .

Some discussion occurred on the CR. Should we detect future and spare anywhere in the hostname? I was under the impression that this is not normal nor should we entrench it as a standard practice by checking for it. (The case of spare being inside the hostname turned out to be, essentially, in error afair).

Wed, Nov 20, 4:12 AM · Patch-For-Review, netbox