Page MenuHomePhabricator

CDanis (Chris Danis)
SRE

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Nov 5 2018, 2:54 PM (53 w, 1 d)
Availability
Available
IRC Nick
cdanis
LDAP User
CDanis
MediaWiki User
CDanis (WMF) [ Global Accounts ]

Recent Activity

Yesterday

CDanis added a parent task for T211538: Report cpu seconds spent from MediaWiki to Graphite: Unknown Object (Task).
Tue, Nov 12, 7:27 PM · Performance-Team

Fri, Nov 8

CDanis added a comment to T234450: Some Special:Contributions requests cause "Error: 0" from database or WMFTimeoutException.

+1 to @tstarling's proposal.

Fri, Nov 8, 3:47 PM · User-notice, Core Platform Team Workboards (Clinic Duty Team), Vuln-DoS, Security, Performance Issue, MediaWiki-Special-pages, Wikimedia-production-error

Wed, Nov 6

CDanis claimed T237424: stunnel-wrap all rsync::server usage.

Generated a list of all $hosts_allow arguments from rsync::server::module invocations across all of Puppet: P9544

Wed, Nov 6, 9:21 PM · Operations
CDanis updated the title for P9544 ✔️ cdanis@puppetdb1001.eqiad.wmnet ~/resources 🕓🍵 jq '.[] | select(.type == "Rsync::Server::Module") | {"node": .certname, "title": .title, "hosts_allow":.parameters.hosts_allow}' * | phaste from Masterwork From Distant Lands to ✔️ cdanis@puppetdb1001.eqiad.wmnet ~/resources 🕓🍵 jq '.[] | select(.type == "Rsync::Server::Module") | {"node": .certname, "title": .title, "hosts_allow":.parameters.hosts_allow}' * | phaste.
Wed, Nov 6, 8:48 PM
CDanis added a comment to T237424: stunnel-wrap all rsync::server usage.

Another thing that just came up: not all users of rsync::server::module are actually passing an array to the $hosts_allow argument: https://gerrit.wikimedia.org/r/c/operations/puppet/+/549142
Need to go through PuppetDB and look for this.

Wed, Nov 6, 6:39 PM · Operations
CDanis updated the task description for T237259: Document all uses of the puppetCA certificate.
Wed, Nov 6, 6:10 PM · Patch-For-Review, DBA, User-jbond, Puppet, Operations
CDanis updated the task description for T237259: Document all uses of the puppetCA certificate.
Wed, Nov 6, 6:09 PM · Patch-For-Review, DBA, User-jbond, Puppet, Operations
ema awarded T237424: stunnel-wrap all rsync::server usage a Baby Tequila token.
Wed, Nov 6, 10:40 AM · Operations
fgiunchedi awarded T237424: stunnel-wrap all rsync::server usage a Like token.
Wed, Nov 6, 10:19 AM · Operations

Tue, Nov 5

CDanis added a comment to T237424: stunnel-wrap all rsync::server usage.

There's also some added logspam to be fixed:

Tue, Nov 5, 10:36 PM · Operations
CDanis added a comment to T237424: stunnel-wrap all rsync::server usage.

After far too many things that were subtly wrong, it works!

1✔️ cdanis@netmon2001.wikimedia.org ~ 🕔🍺 sudo /usr/local/sbin/sync-var-lib-smokeping
22019.11.05 22:12:16 LOG5[ui]: stunnel 5.39 on x86_64-pc-linux-gnu platform
32019.11.05 22:12:16 LOG5[ui]: Compiled with OpenSSL 1.1.0c 10 Nov 2016
42019.11.05 22:12:16 LOG5[ui]: Running with OpenSSL 1.1.0l 10 Sep 2019
52019.11.05 22:12:16 LOG5[ui]: Update OpenSSL shared libraries or rebuild stunnel
62019.11.05 22:12:16 LOG5[ui]: Threading:PTHREAD Sockets:POLL,IPv6,SYSTEMD TLS:ENGINE,FIPS,OCSP,PSK,SNI Auth:LIBWRAP
72019.11.05 22:12:16 LOG5[ui]: Reading configuration from file /tmp/sync-ssl-wrapper.stunnel.conf.6NZf661Q
82019.11.05 22:12:16 LOG5[ui]: UTF-8 byte order mark not detected
92019.11.05 22:12:16 LOG5[ui]: FIPS mode disabled
102019.11.05 22:12:16 LOG4[ui]: Service [stunnel] uses "verify = 2" without subject checks
112019.11.05 22:12:16 LOG4[ui]: Use "checkHost" or "checkIP" to restrict trusted certificates
122019.11.05 22:12:16 LOG5[ui]: Configuration successful
132019.11.05 22:12:16 LOG5[0]: Service [stunnel] accepted connection from unnamed socket
142019.11.05 22:12:16 LOG5[0]: s_connect: connected 208.80.154.5:1873
152019.11.05 22:12:16 LOG5[0]: Service [stunnel] connected remote server from 208.80.153.110:50364
162019.11.05 22:12:16 LOG5[0]: Certificate accepted at depth=0: CN=netmon1002.wikimedia.org
172019.11.05 22:12:17 LOG5[0]: Connection closed: 486490 byte(s) sent to TLS, 1472301 byte(s) sent to socket

Tue, Nov 5, 10:13 PM · Operations
CDanis created P9532 (An Untitled Masterwork).
Tue, Nov 5, 10:12 PM
CDanis created T237424: stunnel-wrap all rsync::server usage.
Tue, Nov 5, 3:25 PM · Operations
CDanis updated subscribers of T237407: basic prometheus monitoring for PoolCounter.
Tue, Nov 5, 2:08 PM · Operations, observability, serviceops
CDanis created T237407: basic prometheus monitoring for PoolCounter.
Tue, Nov 5, 2:05 PM · Operations, observability, serviceops

Mon, Nov 4

CDanis edited P9527 Masterwork From Distant Lands.
Mon, Nov 4, 7:48 PM
CDanis created P9526 (An Untitled Masterwork).
Mon, Nov 4, 7:39 PM
CDanis updated the title for P9523 locking process tree: systemd---sshd---sshd---sshd(cdanis)---zsh---sudo(root)---locktest.sh---sleep from untitled to locking process tree: systemd---sshd---sshd---sshd(cdanis)---zsh---sudo(root)---locktest.sh---sleep.
Mon, Nov 4, 6:34 PM
CDanis created P9523 locking process tree: systemd---sshd---sshd---sshd(cdanis)---zsh---sudo(root)---locktest.sh---sleep.
Mon, Nov 4, 6:32 PM
CDanis added a comment to T236277: Extend Puppet CA Expiry date .

In terms of identifying services that use keys issued by the puppet CA -- is it wrong to think that the following would be a complete list?

  • keys created using cergen
  • users of base::expose_puppet_certs
  • the few users we have that are referencing either puppet_ssldir() or manually hardcoding the /var/lib/puppet/ssl directory
Mon, Nov 4, 2:23 PM · DBA, Patch-For-Review, User-jbond, Puppet, Operations

Fri, Nov 1

CDanis closed T237134: Heavy traffic on ORES CODFW (2019-11-01 @ 1900 UTC) as Resolved.
Fri, Nov 1, 9:21 PM · Patch-For-Review, ORES, Scoring-platform-team (Current)
CDanis updated the task description for T215301: codfw spare pool system for partman testing.
Fri, Nov 1, 2:39 PM · Operations, hardware-requests

Wed, Oct 30

CDanis archived P9503 Masterwork From Distant Lands.
Wed, Oct 30, 8:29 PM
CDanis edited P9503 Masterwork From Distant Lands.
Wed, Oct 30, 8:29 PM
CDanis closed T231870: Add new Graphite instance in Grafana as Resolved.

Okay, I think this is done and working!

Wed, Oct 30, 1:55 PM · Patch-For-Review, observability, Performance-Team
CDanis closed T231870: Add new Graphite instance in Grafana, a subtask of T225416: Unify synthetic testing scripts, as Resolved.
Wed, Oct 30, 1:55 PM · Patch-For-Review, WebPageReplay, WebPageTest, Performance-Team

Tue, Oct 29

CDanis edited P9496 (An Untitled Masterwork).
Tue, Oct 29, 3:14 PM
CDanis created P9496 (An Untitled Masterwork).
Tue, Oct 29, 2:44 PM

Mon, Oct 28

CDanis created T236744: track NIC firmware version numbers across the fleet.
Mon, Oct 28, 9:21 PM · Patch-For-Review, Operations, Traffic
CDanis triaged T236700: 'LVS connections' graph on Load Balancers dashboard takes a rate of a gauge as Normal priority.
Mon, Oct 28, 3:40 PM · Traffic, observability, Operations
CDanis created T236700: 'LVS connections' graph on Load Balancers dashboard takes a rate of a gauge.
Mon, Oct 28, 3:36 PM · Traffic, observability, Operations
CDanis updated the task description for T220838: Upgrade grafana to 6.x.
Mon, Oct 28, 3:26 PM · Performance-Team (Radar), observability, Operations
CDanis closed T234567: global HTTP (un)availability number, as reported in Frontend Traffic dashboard, is bogus as Resolved.
Mon, Oct 28, 3:08 PM · Traffic, observability, Operations

Sun, Oct 27

CDanis added a comment to T234450: Some Special:Contributions requests cause "Error: 0" from database or WMFTimeoutException.

A community member noticed this today; seen in #wikimedia-tech on IRC:

12:24:33	<Nemo_bis>	Since when a limit > 500 no longer works in Special:Contributions? https://it.wikipedia.org/w/index.php?title=Speciale:Contributi&limit=550&target=Nemo+bis
Sun, Oct 27, 4:09 PM · User-notice, Core Platform Team Workboards (Clinic Duty Team), Vuln-DoS, Security, Performance Issue, MediaWiki-Special-pages, Wikimedia-production-error

Sat, Oct 26

CDanis created T236598: cr3-esams crash.
Sat, Oct 26, 4:22 PM · Operations, netops
CDanis edited P9483 ripe-atlas-eqsin IPv6 noise.
Sat, Oct 26, 2:49 PM
CDanis updated the title for P9483 ripe-atlas-eqsin IPv6 noise from untitled to ripe-atlas-eqsin IPv6 noise.
Sat, Oct 26, 2:14 PM
CDanis created P9483 ripe-atlas-eqsin IPv6 noise.
Sat, Oct 26, 2:14 PM

Fri, Oct 25

CDanis added a comment to T236486: Cannot access production replica database from mwmaint1002.

@Reedy that's the wrong place to look nowadays. The two good places to look for production database configs:
https://noc.wikimedia.org/db.php
https://noc.wikimedia.org/dbconfig/eqiad.json

Fri, Oct 25, 2:06 PM · Operations

Thu, Oct 24

CDanis claimed T231870: Add new Graphite instance in Grafana.
Thu, Oct 24, 8:16 PM · Patch-For-Review, observability, Performance-Team
CDanis created P9473 (An Untitled Masterwork).
Thu, Oct 24, 3:11 PM

Wed, Oct 23

CDanis added a comment to T236104: Cache of wmf-config/InitialiseSettings often 1 step behind.

How reliable is the filemtime function in a scap world? Does scap / rsync always migrate the mtime from the host to the target machine?

Wed, Oct 23, 4:36 PM · Deployments, Release-Engineering-Team, Performance-Team (Radar), serviceops
CDanis updated the task description for T234450: Some Special:Contributions requests cause "Error: 0" from database or WMFTimeoutException.
Wed, Oct 23, 1:34 PM · User-notice, Core Platform Team Workboards (Clinic Duty Team), Vuln-DoS, Security, Performance Issue, MediaWiki-Special-pages, Wikimedia-production-error

Tue, Oct 22

CDanis added a comment to T234450: Some Special:Contributions requests cause "Error: 0" from database or WMFTimeoutException.

@Reedy Can you elaborate? The only rate-limiting I'm aware of is the simultaneous edit limiting provided by PoolCounter, and the limited support for limiting simultaneous requests from the same IP address in Varnish.

Tue, Oct 22, 8:57 PM · User-notice, Core Platform Team Workboards (Clinic Duty Team), Vuln-DoS, Security, Performance Issue, MediaWiki-Special-pages, Wikimedia-production-error
CDanis added a comment to T234450: Some Special:Contributions requests cause "Error: 0" from database or WMFTimeoutException.

The bug was turned into a private issue because of potential DoS concerns: a relatively low volume of these long-date-range limit=5000 Special:Contributions queries (a volume that I could trivially send from my laptop, for instance) create sufficient load to cause a great deal of hurt for our database servers.

Tue, Oct 22, 6:01 PM · User-notice, Core Platform Team Workboards (Clinic Duty Team), Vuln-DoS, Security, Performance Issue, MediaWiki-Special-pages, Wikimedia-production-error
CDanis updated subscribers of T236104: Cache of wmf-config/InitialiseSettings often 1 step behind.

This happened yesterday, during firefighting work on {T235949}

Tue, Oct 22, 12:26 PM · Deployments, Release-Engineering-Team, Performance-Team (Radar), serviceops

Mon, Oct 21

CDanis added a comment to T222391: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster).

something to note/fix for future migrations: one option for how to push DNS changes when gerrit is down https://tools.wmflabs.org/sal/log/AW3wKZvNfYQT6VcDX2pS

Mon, Oct 21, 9:35 PM · Patch-For-Review, Release-Engineering-Team (Development services), Release-Engineering-Team-TODO, serviceops, Operations, Gerrit
CDanis created P9417 (An Untitled Masterwork).
Mon, Oct 21, 8:35 PM
CDanis created P9416 (An Untitled Masterwork).
Mon, Oct 21, 8:35 PM
CDanis updated the task description for T220838: Upgrade grafana to 6.x.
Mon, Oct 21, 2:01 PM · Performance-Team (Radar), observability, Operations

Fri, Oct 18

CDanis added a comment to T231870: Add new Graphite instance in Grafana.

Yes, sorry for the delay -- I was out a few days and then some other stuff
took priority. I'll try to get it done by next week.

Fri, Oct 18, 5:05 PM · Patch-For-Review, observability, Performance-Team

Thu, Oct 17

CDanis created P9383 (An Untitled Masterwork).
Thu, Oct 17, 8:42 PM
CDanis awarded T224033: Fix operations/puppet.git "rebase hell" a Love token.
Thu, Oct 17, 2:44 PM · Release-Engineering-Team (Development services), Gerrit, Release-Engineering-Team-TODO, Continuous-Integration-Config, Operations

Wed, Oct 16

CDanis updated the task description for T235711: [1.35.0-wmf.2] Excessive "Module {name} not loadable on target mobile " logging from ResourceLoader.
Wed, Oct 16, 9:11 PM · MW-1.35-notes (1.35.0-wmf.2; 2019-10-15), Performance-Team, Operations, MediaWiki-ResourceLoader
CDanis triaged T235711: [1.35.0-wmf.2] Excessive "Module {name} not loadable on target mobile " logging from ResourceLoader as High priority.
Wed, Oct 16, 9:10 PM · MW-1.35-notes (1.35.0-wmf.2; 2019-10-15), Performance-Team, Operations, MediaWiki-ResourceLoader
CDanis created T235711: [1.35.0-wmf.2] Excessive "Module {name} not loadable on target mobile " logging from ResourceLoader.
Wed, Oct 16, 9:10 PM · MW-1.35-notes (1.35.0-wmf.2; 2019-10-15), Performance-Team, Operations, MediaWiki-ResourceLoader
CDanis added a comment to T234450: Some Special:Contributions requests cause "Error: 0" from database or WMFTimeoutException.

How hard would it be to make the limit 500 instead of 5000?

Wed, Oct 16, 3:28 PM · User-notice, Core Platform Team Workboards (Clinic Duty Team), Vuln-DoS, Security, Performance Issue, MediaWiki-Special-pages, Wikimedia-production-error

Tue, Oct 15

CDanis added a comment to T233236: Move labtestwikitech database to clouddb2001-dev.

Hi Andrew!

Tue, Oct 15, 7:52 PM · cloud-services-team (Kanban)
CDanis updated the task description for T235215: Onboarding Reuven Lazarus.
Tue, Oct 15, 4:27 PM · LDAP-Access-Requests, SRE-Access-Requests, Operations
CDanis awarded T235215: Onboarding Reuven Lazarus a Like token.
Tue, Oct 15, 4:26 PM · LDAP-Access-Requests, SRE-Access-Requests, Operations
CDanis added a watcher for Vuln-DoS: CDanis.
Tue, Oct 15, 3:14 PM
CDanis added a comment to T215183: Redundant bootloaders for software RAID.

An update: as of now, 20.8% of the fleet (or 30% of hosts with software RAID enabled) have redundant bootloaders. This is just from fixing the partman configs and waiting for reimages to happen 'naturally'. That's about all the work I'm going to do on this for the time being; I figure improvements will continue as services need to move to Buster.

Tue, Oct 15, 1:36 PM · Operations
CDanis added a comment to T215183: Redundant bootloaders for software RAID.

Buster failed to install on my md install for 2552d12fe15ec1 with "grub install sda sdb failed, cannot install on sda". I cannot be sure it is that change because buster, and because there is weird hybrid software raid + hw raid going on, and it could be a question of using the wrong recipe because of the extra disk, but FYI.

Tue, Oct 15, 1:34 PM · Operations

Oct 12 2019

CDanis added a comment to T226373: Swift object servers become briefly unresponsive on a regular basis.

Is this still an issue?

Oct 12 2019, 10:15 PM · Performance-Team (Radar), User-jijiki, serviceops, Patch-For-Review, SRE-swift-storage, Operations
CDanis created T235334: ErrorException from line 596 of /srv/mediawiki/php-1.35.0-wmf.1/includes/api/ApiQueryBase.php: PHP Notice: Undefined property: stdClass::$page_namespace.
Oct 12 2019, 3:45 PM · MW-1.35-notes (1.35.0-wmf.1; 2019-10-08), Patch-For-Review, MW-1.34-release, Core Platform Team, MediaWiki-API, Operations, Wikimedia-production-error

Oct 11 2019

CDanis created U24 SRE team goals.
Oct 11 2019, 6:07 PM
CDanis added a comment to T234450: Some Special:Contributions requests cause "Error: 0" from database or WMFTimeoutException.

Turns out, we are correctly capping the limit already!
Note LIMIT 5001 at the end.

Oct 11 2019, 12:41 PM · User-notice, Core Platform Team Workboards (Clinic Duty Team), Vuln-DoS, Security, Performance Issue, MediaWiki-Special-pages, Wikimedia-production-error

Oct 10 2019

CDanis committed rLPRI7890ac926d17: secret: varnish: dummy acl bot_blocked_nets (authored by CDanis).
secret: varnish: dummy acl bot_blocked_nets
Oct 10 2019, 8:20 PM

Oct 9 2019

CDanis added a comment to T224033: Fix operations/puppet.git "rebase hell".

Was this discussed during the Monday meeting? What was the outcome?

Oct 9 2019, 5:55 PM · Release-Engineering-Team (Development services), Gerrit, Release-Engineering-Team-TODO, Continuous-Integration-Config, Operations
CDanis added a comment to T234567: global HTTP (un)availability number, as reported in Frontend Traffic dashboard, is bogus.

We probably want to let the new recording rule accumulate some data -- a week's worth? -- and then start using it in the dashboard

Oct 9 2019, 4:46 PM · Traffic, observability, Operations

Oct 7 2019

fgiunchedi awarded T234567: global HTTP (un)availability number, as reported in Frontend Traffic dashboard, is bogus a Love token.
Oct 7 2019, 8:24 PM · Traffic, observability, Operations

Oct 4 2019

CDanis added a comment to T229686: #dbctl: manage 'externalLoads' data.

@Krinkle @aaron -- could one of you please find some time soon to make sure I'm understanding things correctly as I state them below?

Oct 4 2019, 5:17 PM · Performance-Team, DBA, conftool
CDanis added a comment to T229686: #dbctl: manage 'externalLoads' data.

@Marostegui please give me some input on the UI -- what would you like dbctl to call the section that will represent "everything that refers to $wmgOldExtTemplate"? For now I'm thinking cluster1, does that sound good to you? (So you would write something like dbctl instance es1018 depool --section cluster1)

Oct 4 2019, 5:14 PM · Performance-Team, DBA, conftool
CDanis added a comment to T231870: Add new Graphite instance in Grafana.

Sorry, I just realized that this won't work as-is; we don't allow outbound HTTP from production, except via a few proxy servers.

Oct 4 2019, 4:52 PM · Patch-For-Review, observability, Performance-Team
CDanis added a comment to T231870: Add new Graphite instance in Grafana.

Sorry, I just realized that this won't work as-is; we don't allow outbound HTTP from production, except via a few proxy servers.

Oct 4 2019, 4:52 PM · Patch-For-Review, observability, Performance-Team
CDanis moved T233680: No way to cancel a confctl edit invocation from in progress to Backlog on the conftool board.
Oct 4 2019, 3:31 PM · conftool

Oct 3 2019

CDanis triaged T234567: global HTTP (un)availability number, as reported in Frontend Traffic dashboard, is bogus as Normal priority.
Oct 3 2019, 8:35 PM · Traffic, observability, Operations
CDanis updated the task description for T234567: global HTTP (un)availability number, as reported in Frontend Traffic dashboard, is bogus.
Oct 3 2019, 8:35 PM · Traffic, observability, Operations
CDanis created T234567: global HTTP (un)availability number, as reported in Frontend Traffic dashboard, is bogus.
Oct 3 2019, 8:30 PM · Traffic, observability, Operations
CDanis triaged T234533: gerrit shows merged patch as pending as High priority.
Oct 3 2019, 3:03 PM · Release-Engineering-Team-TODO, Gerrit
CDanis created T234533: gerrit shows merged patch as pending.
Oct 3 2019, 3:03 PM · Release-Engineering-Team-TODO, Gerrit

Oct 2 2019

CDanis added a comment to T231870: Add new Graphite instance in Grafana.

Yup, that's part of the datasource definition -- a simple access: proxy stanza

Oct 2 2019, 4:20 PM · Patch-For-Review, observability, Performance-Team
CDanis added a project to T231870: Add new Graphite instance in Grafana: observability.
Oct 2 2019, 2:51 PM · Patch-For-Review, observability, Performance-Team
CDanis added a comment to T231870: Add new Graphite instance in Grafana.

Adding datasources via the UI is disabled, as all datsource definitions are Puppetized in our installation.

Oct 2 2019, 2:50 PM · Patch-For-Review, observability, Performance-Team

Oct 1 2019

CDanis closed T234332: ensure additional puppetmaster files are managed by puppet as Resolved.
Oct 1 2019, 2:53 PM · User-jbond, cloud-services-team, Patch-For-Review, Puppet
CDanis closed T234332: ensure additional puppetmaster files are managed by puppet, a subtask of T234315: upgrade puppet master frontends servers, as Resolved.
Oct 1 2019, 2:53 PM · Patch-For-Review, Puppet
CDanis created T234335: Telia IC-314534 (eqord/eqdfw 10Gbps wave) down.
Oct 1 2019, 2:50 PM · Operations, netops
CDanis closed T233679: dbctl doesn't always correctly translate section names in its output as Resolved.
Oct 1 2019, 2:21 PM · conftool
CDanis committed rOSCT32e5becc83d2: dbctl schema: disallow 's3' value when appropriate (authored by CDanis).
dbctl schema: disallow 's3' value when appropriate
Oct 1 2019, 2:17 PM

Sep 28 2019

CDanis merged T234115: db1070 (s5 master) SMART-reported impending drive failure into T208323: Predictive failures on disk S.M.A.R.T. status.
Sep 28 2019, 5:02 AM · Operations, DBA
CDanis merged task T234115: db1070 (s5 master) SMART-reported impending drive failure into T208323: Predictive failures on disk S.M.A.R.T. status.
Sep 28 2019, 5:02 AM · ops-eqiad, Operations, DBA
CDanis created T234115: db1070 (s5 master) SMART-reported impending drive failure.
Sep 28 2019, 5:01 AM · ops-eqiad, Operations, DBA

Sep 26 2019

CDanis committed rOSCT211499791fc9: dbctl test fixtures: one schema to rule them all (authored by CDanis).
dbctl test fixtures: one schema to rule them all
Sep 26 2019, 7:58 PM
CDanis added a comment to T226044: Prepare Phame to support heavy traffic for a Tech Department blog.

@mmodell @ema After a discussion with TechEng and SRE and the rest of tech-mgt this week in Portland, we are going to hold off on this work (phame as techblog) and instead wait for guidance from TechEng as they have a plan in place and people dedicated to the work. Basically this is going to be rolled into TechEng work on our communication and social plans from a wikimedia tech perspective.

Sep 26 2019, 7:52 PM · Release-Engineering-Team-TODO, User-greg, Release-Engineering-Team (Development services), Operations, Traffic, Phabricator
CDanis moved T229686: #dbctl: manage 'externalLoads' data from Backlog to in progress on the conftool board.
Sep 26 2019, 5:17 PM · Performance-Team, DBA, conftool
CDanis moved T229676: #dbctl: generate hostsByName section as well from Backlog to in progress on the conftool board.
Sep 26 2019, 5:17 PM · conftool
CDanis added a comment to T233680: No way to cancel a confctl edit invocation.

checking the return value of $EDITOR is released but I think there's more work to do here

Sep 26 2019, 5:17 PM · conftool
CDanis closed T229677: #dbctl: add 'comment'/'description' metadata to instances as Resolved.

released

Sep 26 2019, 5:16 PM · DBA, conftool
CDanis closed T231871: dbctl config commit !log message is identical whether the operation succeeded or failed as Resolved.

released

Sep 26 2019, 5:16 PM · conftool
CDanis created T233973: remove service objects from etcd and update documentation.
Sep 26 2019, 5:07 PM · Operations, conftool