Volans (Riccardo Coccioli)
Operations Software Engineer

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Feb 10 2016, 11:25 AM (157 w, 5 d)
Availability
Available
IRC Nick
volans
LDAP User
Volans
MediaWiki User
RCoccioli (WMF) [ Global Accounts ]

Recent Activity

Thu, Feb 14

Volans closed T199911: Systemd session creation fails under I/O load as Resolved.
Thu, Feb 14, 6:47 PM · Operations, Operations-Software-Development
Volans updated the task description for T205867: Expand Spicerack library and SRE Cookbooks - Q2 2018-19 Goal.
Thu, Feb 14, 5:40 PM · Patch-For-Review, Operations-Software-Development, Operations, Goal
Volans added a parent task for T204789: wmf-auto-reimage tries to remove from Debmonitor even with --new: T205885: Cookbooks: convert wmf-auto-reimage scripts to Cookbooks.
Thu, Feb 14, 5:39 PM · Operations, Operations-Software-Development
Volans added a subtask for T205885: Cookbooks: convert wmf-auto-reimage scripts to Cookbooks: T204789: wmf-auto-reimage tries to remove from Debmonitor even with --new.
Thu, Feb 14, 5:39 PM · Patch-For-Review, Operations-Software-Development
Volans added a comment to T199911: Systemd session creation fails under I/O load.

@fgiunchedi with the bandaid the problem doesn't show up anymore, do we want to keep this open for tracking or it's ok to resolve it?

Thu, Feb 14, 5:34 PM · Operations, Operations-Software-Development
Volans added a comment to T179816: Cumin: create external backend for WMCS Puppet API.

The potential source of the data could be https://tools.wmflabs.org/openstack-browser/puppetclass/

Thu, Feb 14, 5:17 PM · Operations-Software-Development
Volans placed T179816: Cumin: create external backend for WMCS Puppet API up for grabs.
Thu, Feb 14, 5:15 PM · Operations-Software-Development
Volans removed a project from T150560: More verbose messages from service-checker-swagger: Operations-Software-Development.

Removing Operations-Software-Development for now, feel free to re-add if you need anything from us on this.

Thu, Feb 14, 5:13 PM · Core Platform Team Backlog (Watching / External), Patch-For-Review, Services (watching), Operations
Volans lowered the priority of T157001: Puppet compiler: abort on git rebase conflict from Normal to Low.
Thu, Feb 14, 4:55 PM · Operations, Operations-Software-Development
Volans added a comment to T216088: Mapping of servers to stakeholders.

I went ahead and tried the naming convention approach adding a column to that table and adding my Phabricator username there were relevant. I've actually added a link to the phab profile, probably just the name is enough if we specify that those should be Phabricator account names.

Thu, Feb 14, 3:32 PM · Operations
Volans updated subscribers of T216133: Increase visibility of auto-generated tasks for RAID errors.
Thu, Feb 14, 2:44 PM · DC-Ops, Operations, Wikimedia-Incident, cloud-services-team (Kanban)
Volans updated subscribers of T216133: Increase visibility of auto-generated tasks for RAID errors.
Thu, Feb 14, 2:44 PM · DC-Ops, Operations, Wikimedia-Incident, cloud-services-team (Kanban)

Wed, Feb 13

Volans closed T211692: Duplicate definitions found in Icinga configuration as Resolved.
Wed, Feb 13, 6:54 PM · Patch-For-Review, Operations, Graphite, monitoring
Volans added a comment to T215892: Degraded RAID on cloudvirt1024.

Sorry I have to amend what I said above, both PD0 and PD3 are missing. I'm sending a patch to improve the get-raid-status-megacli script. With it the new output would have been:

Wed, Feb 13, 2:46 PM · cloud-services-team (Kanban), ops-eqiad, Operations
Volans added a comment to T215892: Degraded RAID on cloudvirt1024.

It seems that PD3 is totally gone, from sudo /usr/local/lib/nagios/plugins/get-raid-status-megacli -a:

Wed, Feb 13, 1:48 PM · cloud-services-team (Kanban), ops-eqiad, Operations
Volans merged T215718: Degraded RAID on cloudvirt1024 into T215892: Degraded RAID on cloudvirt1024.
Wed, Feb 13, 1:32 PM · cloud-services-team (Kanban), ops-eqiad, Operations
Volans merged task T215718: Degraded RAID on cloudvirt1024 into T215892: Degraded RAID on cloudvirt1024.
Wed, Feb 13, 1:32 PM · cloud-services-team, ops-eqiad, Operations
Volans added a project to T215892: Degraded RAID on cloudvirt1024: cloud-services-team.
Wed, Feb 13, 1:25 PM · cloud-services-team (Kanban), ops-eqiad, Operations
Volans added a comment to T216004: Degraded RAID on cloudvirt1018.

I've merged the other auto-generated tasks into this one. FYI, as you can see on some of the others (where NRPE didn't timeout) the RAID was rebuilding.

Wed, Feb 13, 1:22 PM · cloud-services-team (Kanban), ops-eqiad, Operations
Volans merged T215997: Degraded RAID on cloudvirt1018 into T216004: Degraded RAID on cloudvirt1018.
Wed, Feb 13, 1:21 PM · cloud-services-team (Kanban), ops-eqiad, Operations
Volans merged task T215997: Degraded RAID on cloudvirt1018 into T216004: Degraded RAID on cloudvirt1018.
Wed, Feb 13, 1:21 PM · ops-eqiad, Operations
Volans merged T215996: Degraded RAID on cloudvirt1018 into T216004: Degraded RAID on cloudvirt1018.
Wed, Feb 13, 1:21 PM · cloud-services-team (Kanban), ops-eqiad, Operations
Volans merged task T215996: Degraded RAID on cloudvirt1018 into T216004: Degraded RAID on cloudvirt1018.
Wed, Feb 13, 1:21 PM · ops-eqiad, Operations
Volans merged T215994: Degraded RAID on cloudvirt1018 into T216004: Degraded RAID on cloudvirt1018.
Wed, Feb 13, 1:21 PM · cloud-services-team (Kanban), ops-eqiad, Operations
Volans merged task T215994: Degraded RAID on cloudvirt1018 into T216004: Degraded RAID on cloudvirt1018.
Wed, Feb 13, 1:21 PM · ops-eqiad, Operations
Volans merged T215989: Degraded RAID on cloudvirt1018 into T216004: Degraded RAID on cloudvirt1018.
Wed, Feb 13, 1:20 PM · cloud-services-team (Kanban), ops-eqiad, Operations
Volans merged task T215989: Degraded RAID on cloudvirt1018 into T216004: Degraded RAID on cloudvirt1018.
Wed, Feb 13, 1:20 PM · ops-eqiad, Operations
Volans added a comment to T205897: Netbox: fill network topology.

The plan is to then delete the existing cables and import them again with all their attributes.

Wed, Feb 13, 12:00 PM · Operations
Volans added a comment to T214760: icinga1001 crashed.

@RobH thanks a lot for the tests and follow up!
Just to clarify a detail regarding the "normal" reboot messages, at least in a couple of occasions (for the others I'm not fully sure), the host rebooted itself (triggered either by mgmt board or kernel) but was not manually rebooted by us.

Wed, Feb 13, 8:04 AM · Patch-For-Review, ops-eqiad, monitoring, Operations

Tue, Feb 12

Volans added a comment to T214760: icinga1001 crashed.

OK, but IMO let's keep it passive. icinga can continue to run on icinga2001 for now.

Tue, Feb 12, 10:14 PM · Patch-For-Review, ops-eqiad, monitoring, Operations
Volans added a comment to T214760: icinga1001 crashed.

@RobH I actually disagree as the host has crashed already 2 times before it was even in production, so without any icinga-related load, than at least once a couple of months ago and I think 4 times in the last 3 weeks, so I wouldn't call it stable at all 😉

Tue, Feb 12, 10:10 PM · Patch-For-Review, ops-eqiad, monitoring, Operations
jijiki awarded T214760: icinga1001 crashed a Party Time token.
Tue, Feb 12, 12:18 AM · Patch-For-Review, ops-eqiad, monitoring, Operations

Mon, Feb 11

Volans added a comment to T214760: icinga1001 crashed.

Icinga was failovered to icinga2001, @Cmjohnson, @RobH we can proceed either to check if the CPU is properly mounted and/or try to get some replacement parts based on current evidence.

Mon, Feb 11, 11:35 PM · Patch-For-Review, ops-eqiad, monitoring, Operations
Volans added a comment to T215744: Grafana shows zero EventLogging events for around 44 hours around January 15.

Isn't this due to the PDU issue we had that affected prometheus1003?
See https://wikitech.wikimedia.org/wiki/Incident_documentation/20190115-PDU-fuses-blown

Mon, Feb 11, 6:11 PM · monitoring, Analytics

Sun, Feb 10

Volans added a comment to T214760: icinga1001 crashed.

The existing downtimes would be lost?

Sun, Feb 10, 8:45 PM · Patch-For-Review, ops-eqiad, monitoring, Operations
Volans added a comment to T215569: mw1299 is down (jobrunner-canary, now up but depooled).

The host already re-crashed, I'm leaving it as is for now. I've ack'ed the alerts on icinga.

Sun, Feb 10, 8:14 PM · ops-eqiad, Operations
Volans added a comment to T215569: mw1299 is down (jobrunner-canary, now up but depooled).

The host is stuck again (no ping, no ssh, nothing in console but [ 2451.381422] m, nothing new on getsel or getraclog, forcing a reboot,

Sun, Feb 10, 7:57 PM · ops-eqiad, Operations
Volans updated subscribers of T214760: icinga1001 crashed.

As an additional datapoint from T210108, that I've just merged into this task, is that we had 2 reboots that showed the same symptoms during provisioning, I think even before the Icinga software was running.

Sun, Feb 10, 7:43 PM · Patch-For-Review, ops-eqiad, monitoring, Operations
Volans merged task T210108: icinga1001 mysterious reboots into T214760: icinga1001 crashed.
Sun, Feb 10, 7:40 PM · ops-eqiad, Operations, DC-Ops
Volans merged T210108: icinga1001 mysterious reboots into T214760: icinga1001 crashed.
Sun, Feb 10, 7:40 PM · Patch-For-Review, ops-eqiad, monitoring, Operations
Volans added a comment to T210108: icinga1001 mysterious reboots.

I'm merging this with T214760 as those are now clearly just two different manifestation of the same issue (stuck and reboot) and we have the same entries in getraclog:

Sun, Feb 10, 7:39 PM · ops-eqiad, Operations, DC-Ops
Volans added a project to T214760: icinga1001 crashed: ops-eqiad.

icinga1001 was stuck again today, but in a slightly different way that gave us some additional information.
No ping, no ssh and no icinga web were working and no racadm errors were logged, but attaching myself to the console, although unable to get a prompt, I was able to capture the following, that was repeated every few seconds:

Sun, Feb 10, 7:32 PM · Patch-For-Review, ops-eqiad, monitoring, Operations
Volans added a comment to T196336: Icinga passive checks go awal and downtime stops working.

I got also an email alert from our external monitoring today and upon checking icinga1001 I noticed that the uptime coincide with a reboot around that time. See T214760#4941030 for more details. Unclear if related at this point.

Sun, Feb 10, 12:35 AM · Operations, Icinga, monitoring
Volans added a comment to T214760: icinga1001 crashed.

The host crashed again today and got rebooted, nothing in getsel and from getraclog I just got:

Sun, Feb 10, 12:35 AM · Patch-For-Review, ops-eqiad, monitoring, Operations

Tue, Feb 5

Volans removed a project from T212395: cergen CI fails to run on Debian Stretch because cryptography dependency cannot be built against newer openssl version: Operations-Software-Development.
Tue, Feb 5, 1:39 AM · Continuous-Integration-Config, Operations

Mon, Feb 4

Volans added a comment to T208861: cumin: Support multiple OpenStack regions.

I'd like to add that you should think about it in the general sense of any openstack installation, not only ours. Cumin is a generic software that has no WMF-specific code and we'd like to keep it that way.
That's why I strongly think that we should keep it agnostic and allow the possibility to specify the region in the query.

Mon, Feb 4, 8:45 PM · Patch-For-Review, Operations-Software-Development

Sat, Jan 26

Volans created T214760: icinga1001 crashed.
Sat, Jan 26, 11:27 AM · Patch-For-Review, ops-eqiad, monitoring, Operations
Volans added a comment to T214314: wmf-auto-reimage-host: icinga downtime error.

Forgot to mention that as a quick fix passing the option --rename-mgmt` explicitely should avoid the problem.

Sat, Jan 26, 1:07 AM · Operations
Volans added a comment to T214314: wmf-auto-reimage-host: icinga downtime error.

The problem was misdiagnosed here, we have two issues, one is related to Icinga and is anyway considered a warning and doesn't make the script stop, the other is the real error instead.

Sat, Jan 26, 12:07 AM · Operations

Fri, Jan 25

Volans removed a project from T214560: [CVE-2017-18342] pyyaml vulnerability in netbox deploy repository: Security.
Fri, Jan 25, 9:23 PM · Security, Upstream, Operations-Software-Development
Volans updated subscribers of T205897: Netbox: fill network topology.

@ayounsi you can test it on the WMCS instance that @crusnov has created to test the upgrade ;) (do not add sensitive data there)

Fri, Jan 25, 5:43 PM · Operations
Volans closed T214560: [CVE-2017-18342] pyyaml vulnerability in netbox deploy repository as Resolved.

With the latest version we're deploying we don't have anymore netmiko as it was coming with napalm and we removed that dependency. So all good here, resolving.

Fri, Jan 25, 1:08 AM · Security, Upstream, Operations-Software-Development

Thu, Jan 24

Volans added a parent task for T202885: Migrate elasticsearch scripts to spicerack cookbooks: T203943: Spicerack cookbooks TODO list.
Thu, Jan 24, 9:51 PM · Patch-For-Review, Discovery-Search, Epic, Operations
Volans added a subtask for T203943: Spicerack cookbooks TODO list: T202885: Migrate elasticsearch scripts to spicerack cookbooks.
Thu, Jan 24, 9:51 PM · Operations-Software-Development, User-jijiki, User-Joe, Operations
Volans added a parent task for T212866: Create Spicerack cook book to drain/reboot/uncordon a Kubernetes worker: T203943: Spicerack cookbooks TODO list.
Thu, Jan 24, 9:48 PM · Kubernetes, Operations, Operations-Software-Development
Volans added a subtask for T203943: Spicerack cookbooks TODO list: T212866: Create Spicerack cook book to drain/reboot/uncordon a Kubernetes worker.
Thu, Jan 24, 9:48 PM · Operations-Software-Development, User-jijiki, User-Joe, Operations
Volans renamed T203943: Spicerack cookbooks TODO list from Convert automation scripts to spicerack cookbooks to Spicerack cookbooks TODO list.
Thu, Jan 24, 9:46 PM · Operations-Software-Development, User-jijiki, User-Joe, Operations
Volans added a parent task for T213401: Create a cookbook to copy data between WDQS servers: T203943: Spicerack cookbooks TODO list.
Thu, Jan 24, 9:44 PM · Patch-For-Review, Discovery-Search (Current work), Operations-Software-Development, Wikidata-Query-Service, Wikidata, Operations
Volans added a subtask for T203943: Spicerack cookbooks TODO list: T213401: Create a cookbook to copy data between WDQS servers.
Thu, Jan 24, 9:44 PM · Operations-Software-Development, User-jijiki, User-Joe, Operations
Volans updated subscribers of T214560: [CVE-2017-18342] pyyaml vulnerability in netbox deploy repository.

@hashar what are you suggesting to do here? 3.13 is the latest upstream release.

Thu, Jan 24, 9:18 AM · Security, Upstream, Operations-Software-Development

Wed, Jan 23

Volans moved T205885: Cookbooks: convert wmf-auto-reimage scripts to Cookbooks from Backlog to In Progress on the Operations-Software-Development board.
Wed, Jan 23, 8:33 PM · Patch-For-Review, Operations-Software-Development

Jan 17 2019

Volans added a project to T211692: Duplicate definitions found in Icinga configuration: Operations.

Bump, this is still happening.

Jan 17 2019, 10:30 AM · Patch-For-Review, Operations, Graphite, monitoring

Jan 15 2019

Volans added a comment to T212418: Memory error on restbase1016.

FYI the host is currently down due to a partial power issue in that rack.

Jan 15 2019, 7:32 PM · Patch-For-Review, Core Platform Team Backlog (Watching / External), Services (watching), RESTBase-Cassandra, RESTBase, Operations

Jan 14 2019

Volans added a comment to T212524: Upgrade Netbox to 2.5.x.

Updated https://wikitech.wikimedia.org/wiki/Netbox with sanitizer script. If anyone has opinions on copying over passwords I'm interested, for the time being i'll blank them all except for mine for convenience.

Jan 14 2019, 9:30 PM · Patch-For-Review, Operations-Software-Development
Volans added a comment to T196628: CI: upgrade tox, currently running 2.6.0.

@hashar thanks a lot, CI is finally running successfully for Debmonitor (i.e. https://gerrit.wikimedia.org/r/c/operations/software/debmonitor/+/483131 )

Jan 14 2019, 2:38 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Continuous-Integration-Infrastructure

Jan 10 2019

Volans added a comment to T213422: es1019 IPMI and its management interface are unresponsive (again).

@jcrespo you could try first any of the known/listed things in https://wikitech.wikimedia.org/wiki/Management_Interfaces (aliased from IPMI) and of course feel free to expand it if it's something different.

Jan 10 2019, 1:13 PM · Patch-For-Review, Operations, ops-eqiad

Jan 9 2019

Volans created T213296: Cumin: batch_sleep is waited after last execution in some cases.
Jan 9 2019, 3:52 PM · Patch-For-Review, Operations-Software-Development

Jan 8 2019

Volans closed T213052: base::monitoring::host's alarm dashboard links are broken as Resolved.

It should be resolved with the above patch, feel free to reopen if not.

Jan 8 2019, 11:44 AM · Patch-For-Review, Icinga, monitoring, Operations

Jan 7 2019

Volans added a comment to T212010: Degraded RAID on sodium.

@Dzahn it's reported as degraded by megacli:

Jan 7 2019, 6:28 PM · ops-eqiad, Operations
Volans claimed T213052: base::monitoring::host's alarm dashboard links are broken .
Jan 7 2019, 4:31 PM · Patch-For-Review, Icinga, monitoring, Operations
Volans triaged T213084: Build an understanding of our needs around external monitoring services - Q3 2018/19 goal as Normal priority.
Jan 7 2019, 3:28 PM · User-CDanis, monitoring, Goal

Jan 5 2019

Volans closed T212969: Degraded RAID alert not acking notifications as Resolved.

Path deployed, resolving for now. Please re-open if that doesn't fix it.

Jan 5 2019, 11:58 AM · Patch-For-Review, Operations, monitoring
Volans added a comment to T212969: Degraded RAID alert not acking notifications.

The raid handler had the old path valid in jessie for the command file.

Jan 5 2019, 11:49 AM · Patch-For-Review, Operations, monitoring
Volans claimed T212969: Degraded RAID alert not acking notifications.
Jan 5 2019, 11:44 AM · Patch-For-Review, Operations, monitoring

Jan 3 2019

Volans created P7957 Example for netbox script.
Jan 3 2019, 7:22 PM
fgiunchedi awarded T212669: icinga doesn't log ampersand in notes_url links a Like token.
Jan 3 2019, 8:42 AM · Patch-For-Review, monitoring, Operations

Jan 2 2019

Volans committed rCUMIN2e58521a999c: tests: test also with Python 3.7 (authored by Volans).
tests: test also with Python 3.7
Jan 2 2019, 8:07 PM
Volans committed rCUMIN8d44ec94c75b: tests: fix Pytest RemovedInPytest4Warning (authored by Volans).
tests: fix Pytest RemovedInPytest4Warning
Jan 2 2019, 8:07 PM
Volans closed T212669: icinga doesn't log ampersand in notes_url links as Resolved.

Fix applied, it required an Icinga restart. See the commit message for more details.

Jan 2 2019, 6:38 PM · Patch-For-Review, monitoring, Operations
Volans claimed T212669: icinga doesn't log ampersand in notes_url links.
Jan 2 2019, 5:48 PM · Patch-For-Review, monitoring, Operations
Volans removed a parent task for T212526: Implement netbox reports which check against PuppetDB: T205868: Expand Netbox usage - Q2 2018-19 Goal.
Jan 2 2019, 4:24 PM · User-crusnov, Operations-Software-Development
Volans removed a subtask for T205868: Expand Netbox usage - Q2 2018-19 Goal: T212526: Implement netbox reports which check against PuppetDB.
Jan 2 2019, 4:24 PM · Operations, Operations-Software-Development, Goal
Volans removed a subtask for T205868: Expand Netbox usage - Q2 2018-19 Goal: T212524: Upgrade Netbox to 2.5.x.
Jan 2 2019, 4:24 PM · Operations, Operations-Software-Development, Goal
Volans removed a parent task for T212524: Upgrade Netbox to 2.5.x: T205868: Expand Netbox usage - Q2 2018-19 Goal.
Jan 2 2019, 4:24 PM · Patch-For-Review, Operations-Software-Development
Volans added a comment to T205900: Cumin: add backend for Netbox.

Removed the parent task as the stretch part of the goal was not done, but keeping the task around as we want to create this backend anyway.

Jan 2 2019, 4:23 PM · Operations, Operations-Software-Development
Volans removed a subtask for T205868: Expand Netbox usage - Q2 2018-19 Goal: T205900: Cumin: add backend for Netbox.
Jan 2 2019, 4:23 PM · Operations, Operations-Software-Development, Goal
Volans removed a parent task for T205900: Cumin: add backend for Netbox: T205868: Expand Netbox usage - Q2 2018-19 Goal.
Jan 2 2019, 4:23 PM · Operations, Operations-Software-Development
Volans removed a parent task for T205897: Netbox: fill network topology: T205868: Expand Netbox usage - Q2 2018-19 Goal.
Jan 2 2019, 4:22 PM · Operations
Volans removed a subtask for T205868: Expand Netbox usage - Q2 2018-19 Goal: T205897: Netbox: fill network topology.
Jan 2 2019, 4:22 PM · Operations, Operations-Software-Development, Goal
Volans closed T205868: Expand Netbox usage - Q2 2018-19 Goal as Resolved.

Resolving as the quarter is over and the goal-specific tasks have been completed.

Jan 2 2019, 4:22 PM · Operations, Operations-Software-Development, Goal
Volans triaged T212783: cumin: allow to suppress output and progress bars as Normal priority.
Jan 2 2019, 2:21 PM · Patch-For-Review, Operations-Software-Development
Volans closed T177385: Upgrade Cumin masters to stretch as Resolved.

Migration has been completed and cumin[12]001 are fully in service since few weeks. Resolving.

Jan 2 2019, 1:58 PM · Operations-Software-Development, Operations

Dec 31 2018

Volans updated subscribers of T212697: uwsgi's logsocket_plugin.so causes segfaults during log rotation.

Thanks @elukey, I'll have a look in the next days, at first look it seems that:

  • the logrotate config is the default one from the uwsgi package and doesn't point to where we actually log and where they are rotated independently (/srv/log/netbox).
  • we don't use options like log-reopen in uwsgi, something to look into to fine tune the configuration so that it doesn't trigger this error.
Dec 31 2018, 5:11 PM · Operations

Dec 29 2018

Volans triaged T212681: /dev/log symlink to /run/systemd/journal/dev-log disappeared on kubernetes1001 as Normal priority.
Dec 29 2018, 12:03 PM · Operations
Volans created T212681: /dev/log symlink to /run/systemd/journal/dev-log disappeared on kubernetes1001.
Dec 29 2018, 12:02 PM · Operations

Dec 24 2018

Volans added a comment to T212526: Implement netbox reports which check against PuppetDB.

If I may it seems much more natural to return a dict hostname: value 😉 (not to mention the past issues with top-level JSON arrays).

Dec 24 2018, 10:39 AM · User-crusnov, Operations-Software-Development

Dec 19 2018

Volans added a comment to T211979: 'provision' Grafana's datasources as YAML in puppet.

@CDanis I have a quick and dirty solution that seems seems to do 95% of the work, given that is a one-shot I think it might be near-usable, just my 2c at it, feel free to ignore ;)

Dec 19 2018, 7:40 PM · Patch-For-Review, monitoring, User-CDanis
Volans added a comment to T205867: Expand Spicerack library and SRE Cookbooks - Q2 2018-19 Goal.

Current status is:

  • ~95% of the library migrated
  • documentation done
  • other wmf-* script partially done (peding merge)
  • wmf-auto-reimage script TODO
Dec 19 2018, 6:34 PM · Patch-For-Review, Operations-Software-Development, Operations, Goal
Volans added a comment to T205886: Cookbooks: convert remaining wmf-* scripts.

Some of the other scripts have been migrated, CR are ready, pending to deploy the new 0.0.10 release of Spicerack.
Given that the newer Spicerack package was not deployed today and we decided to postpone it to the first week after the holidays, I've add -2 to those CRs to avoid accidental merges.

Dec 19 2018, 6:33 PM · Patch-For-Review, Operations-Software-Development
Volans added a comment to T205885: Cookbooks: convert wmf-auto-reimage scripts to Cookbooks.

This is overdue in respect to the Goal, and will be tackled right after the holidays.

Dec 19 2018, 6:32 PM · Patch-For-Review, Operations-Software-Development