Volans (Riccardo Coccioli)
Operations Software Engineer

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Sunday

  • Clear sailing ahead.

User Details

User Since
Feb 10 2016, 11:25 AM (92 w, 2 d)
Availability
Available
IRC Nick
volans
LDAP User
Volans
MediaWiki User
RCoccioli (WMF)

Recent Activity

Today

Volans added a comment to T175708: Add annotations per run in WebPagetest.

To reply to @Peter in T180428#3762974.

Fri, Nov 17, 12:48 PM · Performance-Team, WebPageTest

Yesterday

Volans added a comment to T180724: mw2251 hardware error.

I've ack'ed the Icinga host down alarm with a link to this task

Thu, Nov 16, 9:01 PM · ops-codfw, Operations

Tue, Nov 14

Volans added a comment to T178807: Onboard aborrero to WMF.

I've added @aborrero to racktables.

Tue, Nov 14, 5:16 PM · Patch-For-Review, cloud-services-team
Volans updated the task description for T178807: Onboard aborrero to WMF.
Tue, Nov 14, 5:15 PM · Patch-For-Review, cloud-services-team
Volans added a comment to T180428: Upgrade to latest Grafana 4.6.

@fgiunchedi yes I'm worried about scalability in terms of transactions write rate (if the automated tools will add many annotations in a short period of time and growth of the SQLite database in size, given that is the same that contains all the dashboards metadata.

Tue, Nov 14, 1:58 PM · Performance-Team (Radar), Graphite, monitoring, Operations
Volans updated subscribers of T180428: Upgrade to latest Grafana 4.6.

I personally don't think that using the native annotations available in Grafana 4.6 from an automated system (WebPageTest in this case) is a good idea in our setup, given that it's currently using SQLite as its own database. @fgiunchedi thoughts?

Tue, Nov 14, 10:43 AM · Performance-Team (Radar), Graphite, monitoring, Operations

Wed, Nov 8

Volans merged task T180040: Degraded RAID on lvs3001 into T168619: Degraded RAID on lvs3001.
Wed, Nov 8, 3:15 PM · ops-esams, Operations
Volans merged T180040: Degraded RAID on lvs3001 into T168619: Degraded RAID on lvs3001.
Wed, Nov 8, 3:15 PM · ops-esams, Operations
Volans added a comment to T180023: [DRAFT][RfC] Deployment of python applications in production.

Which deployment method to choose

Wed, Nov 8, 1:48 PM · User-Joe, Operations
This is a test notification, sent at Wed, Nov 8, 12:07.
Wed, Nov 8, 11:07 AM
This is a test notification, sent at Wed, Nov 8, 12:06.
Wed, Nov 8, 11:06 AM

Mon, Nov 6

Volans triaged T179816: Cumin: create external backend for WMCS Puppet API as Normal priority.
Mon, Nov 6, 11:27 AM · Operations-Software-Development
Volans created T179816: Cumin: create external backend for WMCS Puppet API.
Mon, Nov 6, 11:27 AM · Operations-Software-Development
Volans triaged T178279: Cumin: add support for role/profile shortcuts in PuppetDB backend as Normal priority.
Mon, Nov 6, 11:26 AM · Operations-Software-Development
Volans closed T178342: Cumin: add support for external backends as Resolved.
Mon, Nov 6, 11:25 AM · Operations-Software-Development
Volans closed T178279: Cumin: add support for role/profile shortcuts in PuppetDB backend as Resolved.
Mon, Nov 6, 11:25 AM · Operations-Software-Development

Thu, Nov 2

Volans added a comment to T171473: labvirt1015 crashes.

@chasemp FYI if you add the labs project to the cumin query is immediate (as compared to go over all projects) and OpenStack API already does a regex, so the prefix is enough, without any special char. To summarize project:testlabs name:labvirt1015stresstest should do 😉.

Thu, Nov 2, 10:38 PM · cloud-services-team (Kanban), DC-Ops, Operations, ops-eqiad
Volans created T179593: Cumin: upload generated documentation to doc.w.o.
Thu, Nov 2, 3:55 PM · Patch-For-Review, Release-Engineering-Team (Kanban), Operations-Software-Development, Continuous-Integration-Config
Volans updated subscribers of T179395: Cluster puppet variable and ganglia decommission.

As partially discussed in the last monitoring meeting, this is one possibility:

Thu, Nov 2, 12:32 PM · Patch-For-Review, monitoring, Operations

Mon, Oct 30

Volans moved T170353: Icinga: timeseries checks should have the link to a graph with the data from Up next to In progress on the monitoring board.
Mon, Oct 30, 2:56 PM · Patch-For-Review, Operations, monitoring

Sat, Oct 28

Volans added a comment to T179192: Check analytics1037 power supply status.

Icinga is reporting it as critical:

Sensor Type(s) Temperature, Power_Supply Status: Critical [PS Redundancy = Critical, Status = Critical, Status = Critical]
'Inlet Temp'=21.00;3.00:42.00;-7.00:47.00 'Exhaust Temp'=45.00;8.00:70.00;3.00:75.00 'Temp'=64.00 'Temp'=55.00
Sat, Oct 28, 10:55 AM · ops-eqiad, Operations, User-Elukey, Analytics
Volans created T179230: Puppet wmf-style-guide: array of classes not detected properly.
Sat, Oct 28, 10:15 AM · Puppet, Operations

Fri, Oct 27

Volans added a comment to T178553: Support multi-instance hosts on mediawiki-config.

I'm not very familiar with the beta-side of wmf-config, but from a quick look it seems to me that db-labs.php is used there.
To be on the safe side, could we apply this first in deployment-prep adding the :3306 to some of the DBs and verify that everything is still working as expected? Will this test the same code branches that will be used once in production?

Fri, Oct 27, 12:09 PM · Release-Engineering-Team (Watching / External), Performance-Team (Radar), Patch-For-Review, MediaWiki-Platform-Team

Wed, Oct 25

Volans merged T177875: Degraded RAID on bast3002 into T169035: bast3002 sdb broken.
Wed, Oct 25, 3:05 PM · Operations, ops-esams
Volans merged task T177875: Degraded RAID on bast3002 into T169035: bast3002 sdb broken.
Wed, Oct 25, 3:05 PM · ops-esams, Operations
Volans merged task T177881: Degraded RAID on lvs3001 into T168619: Degraded RAID on lvs3001.
Wed, Oct 25, 3:04 PM · ops-esams, Operations
Volans merged T177881: Degraded RAID on lvs3001 into T168619: Degraded RAID on lvs3001.
Wed, Oct 25, 3:04 PM · ops-esams, Operations
Volans moved T179002: Cumin: improve logging from In Progress to In Code Review on the Operations-Software-Development board.
Wed, Oct 25, 3:03 PM · Patch-For-Review, Operations-Software-Development
Volans moved T179002: Cumin: improve logging from Backlog to In Progress on the Operations-Software-Development board.
Wed, Oct 25, 2:52 PM · Patch-For-Review, Operations-Software-Development
Volans created T179002: Cumin: improve logging.
Wed, Oct 25, 2:52 PM · Patch-For-Review, Operations-Software-Development

Tue, Oct 24

Volans added a comment to T178690: Better organization for ops grafana dashboards.

Another idea for better dashboarding: show vertical lines for events other than deployments, e.g. puppet merges

Tue, Oct 24, 12:16 PM · monitoring, Operations
Volans triaged T178877: operations/software repo: flake8 check as Normal priority.
Tue, Oct 24, 10:14 AM · Patch-For-Review, DBA, Operations
Volans created T178877: operations/software repo: flake8 check.
Tue, Oct 24, 10:08 AM · Patch-For-Review, DBA, Operations

Oct 18 2017

Volans added a comment to T84845: improve cron spam visibility.

@Dzahn thanks for pointing this out, I've merged in as duplicate the other task I had opened.

Oct 18 2017, 3:20 PM · monitoring, Operations
Volans added a project to T84845: improve cron spam visibility: monitoring.
Oct 18 2017, 3:20 PM · monitoring, Operations
Volans merged T178311: Cron spam: figure out a way it doesn't get ignored into T84845: improve cron spam visibility.
Oct 18 2017, 3:20 PM · monitoring, Operations
Volans merged task T178311: Cron spam: figure out a way it doesn't get ignored into T84845: improve cron spam visibility.
Oct 18 2017, 3:20 PM · Operations, monitoring
Volans added a comment to T178311: Cron spam: figure out a way it doesn't get ignored.

Closing in favour of T84845

Oct 18 2017, 3:19 PM · Operations, monitoring

Oct 17 2017

Volans moved T178342: Cumin: add support for external backends from In Progress to In Code Review on the Operations-Software-Development board.
Oct 17 2017, 2:47 PM · Operations-Software-Development
Volans closed T159308: Cumin: add man page as Resolved.

Latest release deployed in production, including the manpage.

Oct 17 2017, 2:22 PM · Operations-Software-Development
Volans awarded T178392: Replacement hardware for cumin masters a Like token.
Oct 17 2017, 2:02 PM · hardware-requests, Operations

Oct 16 2017

Volans moved T178342: Cumin: add support for external backends from Backlog to In Progress on the Operations-Software-Development board.
Oct 16 2017, 9:20 PM · Operations-Software-Development
Volans moved T178279: Cumin: add support for role/profile shortcuts in PuppetDB backend from In Progress to In Code Review on the Operations-Software-Development board.
Oct 16 2017, 9:20 PM · Operations-Software-Development
Volans created T178342: Cumin: add support for external backends.
Oct 16 2017, 9:20 PM · Operations-Software-Development
Volans added a comment to T164341: Decommission old memcached hosts - mc1001->mc1018.

@Cmjohnson FYI I'm not using anymore the above hosts for testing.

Oct 16 2017, 5:10 PM · Patch-For-Review, User-Elukey, Operations, ops-eqiad
Volans created T178311: Cron spam: figure out a way it doesn't get ignored.
Oct 16 2017, 3:25 PM · Operations, monitoring
Volans moved T178279: Cumin: add support for role/profile shortcuts in PuppetDB backend from Backlog to In Progress on the Operations-Software-Development board.
Oct 16 2017, 10:16 AM · Operations-Software-Development
Volans created T178279: Cumin: add support for role/profile shortcuts in PuppetDB backend.
Oct 16 2017, 10:16 AM · Operations-Software-Development

Oct 9 2017

Volans changed the status of T157133: Consider adding a --skip-conftool option to puppet-merge from Open to Stalled.

@jcrespo is this still happening?

Oct 9 2017, 9:26 PM · Puppet, Operations-Software-Development, Operations

Oct 8 2017

Volans added a comment to T169680: NFS on dataset1001 overloaded, high load on the hosts that mount it.

We had a re-occurrence of the same, with a very similar stack trace and the same consequences:

[Oct 7 23:27] INFO: task nfsd:16015 blocked for more than 120 seconds.
[  +0.006622]       Tainted: G          I     4.9.0-0.bpo.3-amd64 #1
[  +0.006354] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  +0.007989] nfsd            D    0 16015      2 0x00000000
[  +0.000004]  ffff9f016835c800 0000000000000000 ffff9f0300fa02c0 ffff9f01f477af80
[  +0.000002]  ffff9f032fc187c0 ffffab9a8322f9f0 ffffffffaa40089d 0000000000000000
[  +0.000001]  ffff9f032fdd92c0 000000002fc192c0 ffff9f01f477af80 ffff9f01f477af80
[  +0.000002] Call Trace:
[  +0.000006]  [<ffffffffaa40089d>] ? __schedule+0x23d/0x6d0
[  +0.000002]  [<ffffffffaa400d62>] ? schedule+0x32/0x80
[  +0.000002]  [<ffffffffaa403e79>] ? rwsem_down_write_failed+0x229/0x370
[  +0.000002]  [<ffffffffaa12fa66>] ? __radix_tree_lookup+0x76/0xe0
[  +0.000002]  [<ffffffffaa138593>] ? call_rwsem_down_write_failed+0x13/0x20
[  +0.000002]  [<ffffffffaa403029>] ? down_write+0x29/0x40
[  +0.000056]  [<ffffffffc0856192>] ? xfs_file_buffered_aio_write+0x72/0x2c0 [xfs]
[  +0.000024]  [<ffffffffc08564eb>] ? xfs_file_write_iter+0x10b/0x150 [xfs]
[  +0.000004]  [<ffffffffaa002d60>] ? do_iter_readv_writev+0xb0/0x130
[  +0.000001]  [<ffffffffaa0037bc>] ? do_readv_writev+0x1ac/0x240
[  +0.000024]  [<ffffffffc08563e0>] ? xfs_file_buffered_aio_write+0x2c0/0x2c0 [xfs]
[  +0.000002]  [<ffffffffaa000766>] ? do_dentry_open+0x246/0x300
[  +0.000012]  [<ffffffffc09b1f17>] ? nfsd_vfs_write+0xd7/0x3c0 [nfsd]
[  +0.000005]  [<ffffffffc09b1a38>] ? nfsd_open+0x108/0x1f0 [nfsd]
[  +0.000006]  [<ffffffffc09b4254>] ? nfsd_write+0x124/0x310 [nfsd]
[  +0.000006]  [<ffffffffc09ba1b8>] ? nfsd3_proc_write+0xb8/0x150 [nfsd]
[  +0.000005]  [<ffffffffc09ad2d3>] ? nfsd_dispatch+0xc3/0x250 [nfsd]
[  +0.000020]  [<ffffffffc067c265>] ? svc_process_common+0x475/0x680 [sunrpc]
[  +0.000010]  [<ffffffffc067d2e4>] ? svc_process+0xf4/0x1a0 [sunrpc]
[  +0.000005]  [<ffffffffc09acd29>] ? nfsd+0xe9/0x160 [nfsd]
[  +0.000004]  [<ffffffffc09acc40>] ? nfsd_destroy+0x60/0x60 [nfsd]
[  +0.000003]  [<ffffffffa9e97520>] ? kthread+0xf0/0x110
[  +0.000002]  [<ffffffffa9e2476b>] ? __switch_to+0x2bb/0x700
[  +0.000001]  [<ffffffffa9e97430>] ? kthread_park+0x60/0x60
[  +0.000002]  [<ffffffffaa405875>] ? ret_from_fork+0x25/0x30
Oct 8 2017, 11:24 AM · monitoring, Patch-For-Review, Datasets-General-or-Unknown, Operations

Oct 5 2017

Volans moved T159308: Cumin: add man page from In Progress to In Code Review on the Operations-Software-Development board.
Oct 5 2017, 9:31 PM · Operations-Software-Development

Oct 4 2017

Volans closed T176955: wmf-auto-reimage: add support for renaming while re-imaging as Resolved.
Oct 4 2017, 4:29 PM · Patch-For-Review, Operations-Software-Development
Volans reopened T176955: wmf-auto-reimage: add support for renaming while re-imaging as "Open".
Oct 4 2017, 4:23 PM · Patch-For-Review, Operations-Software-Development
Volans closed T176955: wmf-auto-reimage: add support for renaming while re-imaging as Resolved.
Oct 4 2017, 3:55 PM · Patch-For-Review, Operations-Software-Development
Volans added a project to T177385: Upgrade Cumin masters to stretch: Operations-Software-Development.
Oct 4 2017, 1:45 PM · Operations-Software-Development, Operations
Volans moved T159308: Cumin: add man page from Backlog to In Progress on the Operations-Software-Development board.
Oct 4 2017, 9:58 AM · Operations-Software-Development
Volans moved T167504: New tool to track package updates/status for hosts and images (debmonitor) from Backlog to In Progress on the Operations-Software-Development board.
Oct 4 2017, 9:58 AM · Continuous-Integration-Infrastructure (shipyard), Operations-Software-Development, Operations

Sep 30 2017

Volans closed T176609: Cumin: fine tune WMCS setup as Resolved.

Resolving for now, feel free to re-open it if needed.

Sep 30 2017, 10:03 AM · Cloud-VPS, Operations-Software-Development

Sep 29 2017

Volans added a comment to T165348: Check long-running screen/tmux sessions.

@Dzahn Closed mine, thanks for noticing.

Sep 29 2017, 9:40 PM · Patch-For-Review, monitoring, Operations
Volans closed T177040: On wikitech, [[User:Volans]] is a red link / empty page as Resolved.

Page created 😉

Sep 29 2017, 7:03 AM

Sep 28 2017

Volans moved T176955: wmf-auto-reimage: add support for renaming while re-imaging from In Progress to In Code Review on the Operations-Software-Development board.
Sep 28 2017, 12:09 PM · Patch-For-Review, Operations-Software-Development
Volans moved T176955: wmf-auto-reimage: add support for renaming while re-imaging from Backlog to In Progress on the Operations-Software-Development board.
Sep 28 2017, 12:09 PM · Patch-For-Review, Operations-Software-Development
Volans created T176955: wmf-auto-reimage: add support for renaming while re-imaging.
Sep 28 2017, 12:08 PM · Patch-For-Review, Operations-Software-Development
Volans closed T144264: wmf-reimage and handling of "-n" option as Declined.

The wmf-reimage script has been superseeded by the wmf-auto-reimage scripts, see https://wikitech.wikimedia.org/wiki/Server_Lifecycle#wmf-auto-reimage

Sep 28 2017, 12:07 PM · Operations-Software-Development, Operations
Volans closed T166397: Cumin fails on huge nodelists emitted by its own outputs as Resolved.
Sep 28 2017, 12:06 PM · Operations-Software-Development
Volans added a comment to T166397: Cumin fails on huge nodelists emitted by its own outputs.

With the global grammar available in Cumin, the way to do it now is to use the direct backend syntax D{} inside the query.

Sep 28 2017, 12:06 PM · Operations-Software-Development
Volans added a comment to T165348: Check long-running screen/tmux sessions.

@Dzahn some comments:

  • ms-fe1005 should be whitelisted until T162123 is done
  • I don't think puppetmasters should be whitelisted, all reimage stuff is now in sarin/neodymium only
  • Although reimage stuff is done on sarin/neodymium, they should never reach the threshold IF people close them after the reimage. But I think those two hosts are also used as MySQL management hosts to run long-running tasks, so probably should be whitelisted as per DBAs request.
Sep 28 2017, 8:28 AM · Patch-For-Review, monitoring, Operations

Sep 27 2017

Volans added a comment to T176314: Replace salt on integration and deployment-prep projects.

@hashar want to do the honors of being the first tester? 😉
https://wikitech.wikimedia.org/wiki/Help:Cumin_master

Sep 27 2017, 3:03 PM · RelEng-Archive-FY201718-Q1, Patch-For-Review, Continuous-Integration-Infrastructure, Beta-Cluster-Infrastructure, Technical-Debt, Operations-Software-Development
Volans moved T176314: Replace salt on integration and deployment-prep projects from In Progress to In Code Review on the Operations-Software-Development board.
Sep 27 2017, 1:43 PM · RelEng-Archive-FY201718-Q1, Patch-For-Review, Continuous-Integration-Infrastructure, Beta-Cluster-Infrastructure, Technical-Debt, Operations-Software-Development
Volans moved T176314: Replace salt on integration and deployment-prep projects from Backlog to In Progress on the Operations-Software-Development board.
Sep 27 2017, 1:42 PM · RelEng-Archive-FY201718-Q1, Patch-For-Review, Continuous-Integration-Infrastructure, Beta-Cluster-Infrastructure, Technical-Debt, Operations-Software-Development

Sep 25 2017

Volans moved T176609: Cumin: fine tune WMCS setup from In Progress to In Code Review on the Operations-Software-Development board.
Sep 25 2017, 11:13 AM · Cloud-VPS, Operations-Software-Development
Volans moved T176609: Cumin: fine tune WMCS setup from Backlog to In Progress on the Operations-Software-Development board.
Sep 25 2017, 11:04 AM · Cloud-VPS, Operations-Software-Development
Volans created T176609: Cumin: fine tune WMCS setup.
Sep 25 2017, 11:04 AM · Cloud-VPS, Operations-Software-Development

Sep 24 2017

Volans added a comment to T176573: db2047 got rebooted.

@jcrespo interesting, I guess the documentation in https://wikitech.wikimedia.org/wiki/Platform-specific_documentation/HP_DL3N0#Show_system_event_log_entries needs to be updataed to include the command to show those other logs too. In the event logs there was no event reported before the reboot AFAICT.
There were others referring to the temperature after the reboot too, but the icinga check was ok so I assumed it was a false reading during the reboot process, like other errors that were referring to failed disks. But I might have misunderstood them.

Sep 24 2017, 2:00 PM · ops-codfw, Patch-For-Review, DBA, Operations
Volans triaged T176573: db2047 got rebooted as High priority.
Sep 24 2017, 12:13 PM · ops-codfw, Patch-For-Review, DBA, Operations
Volans created T176573: db2047 got rebooted.
Sep 24 2017, 12:13 PM · ops-codfw, Patch-For-Review, DBA, Operations

Sep 21 2017

Volans closed T174008: Cumin: setup.py installs data_files in wrong directory as Resolved.

Hotfix in debian branch reverted as part of https://gerrit.wikimedia.org/r/#/c/379638/

Sep 21 2017, 11:06 PM · Operations-Software-Development
Volans moved T174008: Cumin: setup.py installs data_files in wrong directory from In Progress to In Code Review on the Operations-Software-Development board.
Sep 21 2017, 11:05 PM · Operations-Software-Development
Volans updated the task description for T164780: Sunset our use of Salt.
Sep 21 2017, 11:05 PM · Patch-For-Review, Goal, Technical-Debt, Operations-Software-Development, Operations
Volans closed T175711: Cumin: create backend for OpenStack as Resolved.
  • OpenStack backend added to Cumin
  • created new Cumin release of Cumin 1.1.0
  • uploaded to wikimedia APT the new release
  • updated Cumin on labpuppetmaster100[1-2]
  • updated Cumin configuration to use the OpenStack backend as default backend
  • updated Wikitech documentation for Cumin (diff here) to include WMCS installation/configuration and the new OpenStack backend
  • see also the OpenStack section in Cumin's README
Sep 21 2017, 11:04 PM · Cloud-VPS, Operations-Software-Development
Volans closed T175711: Cumin: create backend for OpenStack, a subtask of T164780: Sunset our use of Salt, as Resolved.
Sep 21 2017, 11:04 PM · Patch-For-Review, Goal, Technical-Debt, Operations-Software-Development, Operations
Volans updated the task description for T164780: Sunset our use of Salt.
Sep 21 2017, 7:07 PM · Patch-For-Review, Goal, Technical-Debt, Operations-Software-Development, Operations
Volans moved T174008: Cumin: setup.py installs data_files in wrong directory from Backlog to In Progress on the Operations-Software-Development board.
Sep 21 2017, 4:33 PM · Operations-Software-Development
Volans closed T148814: wmf-auto-reimage improvements as Resolved.

All improvements implemented, resolving.

Sep 21 2017, 1:48 PM · Patch-For-Review, Operations-Software-Development
Volans closed T166570: Do something to better handle wmf-reimage runs cleanups/failures as Resolved.

@jcrespo I'm resolving this as resolved given that we merged the new reimage script that doens't use anymore Salt and the wmf-reimage script.
The new one should be more reliable, fail immediately if remote IPMI doesn't work and should not leave running processes.

Sep 21 2017, 1:47 PM · Operations-Software-Development
Volans closed T166570: Do something to better handle wmf-reimage runs cleanups/failures, a subtask of T148814: wmf-auto-reimage improvements, as Resolved.
Sep 21 2017, 1:47 PM · Patch-For-Review, Operations-Software-Development
Volans closed T148817: wmf-auto-reimage: remove dependency on wmf-reimage as Resolved.

Resolved in parent task

Sep 21 2017, 1:43 PM · Operations-Software-Development
Volans closed T148817: wmf-auto-reimage: remove dependency on wmf-reimage, a subtask of T148814: wmf-auto-reimage improvements, as Resolved.
Sep 21 2017, 1:43 PM · Patch-For-Review, Operations-Software-Development
Volans closed T169555: puppet.service systemctl failures after reimage as Resolved.

Resolved in parent task

Sep 21 2017, 1:43 PM · Operations-Software-Development
Volans closed T169555: puppet.service systemctl failures after reimage, a subtask of T148814: wmf-auto-reimage improvements, as Resolved.
Sep 21 2017, 1:43 PM · Patch-For-Review, Operations-Software-Development
Volans closed T149230: wmf-auto-reimage: allow to specify the conftool state as Resolved.

Resolved in parent task

Sep 21 2017, 1:42 PM · Operations-Software-Development
Volans closed T149230: wmf-auto-reimage: allow to specify the conftool state, a subtask of T148814: wmf-auto-reimage improvements, as Resolved.
Sep 21 2017, 1:42 PM · Patch-For-Review, Operations-Software-Development
Volans closed T166300: Remove Salt from wmf-auto-reimage / wmf-reimage as Resolved.
Sep 21 2017, 1:41 PM · Patch-For-Review, Technical-Debt, Operations-Software-Development, Operations
Volans closed T166300: Remove Salt from wmf-auto-reimage / wmf-reimage, a subtask of T148814: wmf-auto-reimage improvements, as Resolved.
Sep 21 2017, 1:41 PM · Patch-For-Review, Operations-Software-Development

Sep 20 2017

Volans added a comment to T176314: Replace salt on integration and deployment-prep projects.

@greg no, that's the right one, cumin it's an additional hashtag of this one ;) Thanks

Sep 20 2017, 9:37 PM · RelEng-Archive-FY201718-Q1, Patch-For-Review, Continuous-Integration-Infrastructure, Beta-Cluster-Infrastructure, Technical-Debt, Operations-Software-Development
Volans moved T175711: Cumin: create backend for OpenStack from In Progress to In Code Review on the Operations-Software-Development board.
Sep 20 2017, 4:21 PM · Cloud-VPS, Operations-Software-Development
Volans closed T175712: Install cumin in the WMCS infrastructure as Resolved.

Cumin master is now installed on labpuppetmaster100[1-2].wikimedia.org and is able to connect via SSH to cloud instances, including the SSH proxy themselves. The targeting for the moment is done just using FQDNs with the direct backend, until T175711 is completed, in the next couple of days.

Sep 20 2017, 11:48 AM · Cloud-VPS, Operations-Software-Development
Volans closed T175712: Install cumin in the WMCS infrastructure, a subtask of T164780: Sunset our use of Salt, as Resolved.
Sep 20 2017, 11:48 AM · Patch-For-Review, Goal, Technical-Debt, Operations-Software-Development, Operations
Volans added a comment to T175712: Install cumin in the WMCS infrastructure.

As a side effect, Beta-Cluster-Infrastructure and Continuous-Integration-Infrastructure would need a way to have a per project cumin master. We don't have access to the WMCS salt master.

The instances are:

deployment-salt02.deployment-prep.eqiad.wmflabs
integration-saltmaster.integration.eqiad.wmflabs

Sep 20 2017, 11:45 AM · Cloud-VPS, Operations-Software-Development

Sep 13 2017

Volans removed a project from T174008: Cumin: setup.py installs data_files in wrong directory: Patch-For-Review.
Sep 13 2017, 4:32 PM · Operations-Software-Development