Page MenuHomePhabricator

dcaro (David Caro)
SRE & amauteur yak shaver

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Mon, Nov 2, 11:59 AM (3 w, 17 h)
Availability
Available
IRC Nick
dcaro
LDAP User
David Caro
MediaWiki User
DCaro (WMF) [ Global Accounts ]

Recent Activity

Yesterday

dcaro added a comment to T267989: Do some checks of how many Quarry queries will break in a multiinstance environment.

And thanks for the shell tip! (found it a bit earlier, and was able to git it working \o/):

Mon, Nov 23, 8:02 PM · Quarry, cloud-services-team (Kanban)
dcaro added a comment to T267989: Do some checks of how many Quarry queries will break in a multiinstance environment.

I'm crossing query.latest_query_rev with query_revision, that shrinks the number quite considerably:

Mon, Nov 23, 7:58 PM · Quarry, cloud-services-team (Kanban)
dcaro added a comment to T267989: Do some checks of how many Quarry queries will break in a multiinstance environment.

I did a preliminary check on all quarry queries, and found 42 that seem to use cross DB (I have to check a bit more the code to see if I'm missing any).

Mon, Nov 23, 3:01 PM · Quarry, cloud-services-team (Kanban)
dcaro added a comment to T267195: CloudVPS: improve VM backups to make them redundant and discoverable.

Mentioned in SAL (#wikimedia-cloud) [2020-11-23T09:57:35Z] <dcaro> Remove control-1 node from the pool (was replaced by control-4) (T267195)

Mon, Nov 23, 9:59 AM · cloud-services-team (Kanban)
dcaro updated the task description for T268438: Toolforge bastion to support python 3.6/3.7/3.8.
Mon, Nov 23, 9:21 AM · Toolforge (Software install/update)
dcaro created T268438: Toolforge bastion to support python 3.6/3.7/3.8.
Mon, Nov 23, 9:18 AM · Toolforge (Software install/update)
dcaro closed D1187: First working version.

landed

Mon, Nov 23, 8:41 AM

Sun, Nov 22

dcaro accepted D1187: First working version.
Sun, Nov 22, 8:26 PM
dcaro requested review of D1187: First working version.
Sun, Nov 22, 8:26 PM
dcaro added a revision to T268418: [sqlchecker] Create a simple service that accepts an sql query and returns if it's cross-db or not: D1187: First working version.
Sun, Nov 22, 8:26 PM · Tool-sqlchecker
dcaro created P13358 Command-Line Input.
Sun, Nov 22, 7:57 PM
dcaro triaged T268418: [sqlchecker] Create a simple service that accepts an sql query and returns if it's cross-db or not as Medium priority.
Sun, Nov 22, 7:15 PM · Tool-sqlchecker
dcaro created T268418: [sqlchecker] Create a simple service that accepts an sql query and returns if it's cross-db or not.
Sun, Nov 22, 7:12 PM · Tool-sqlchecker

Fri, Nov 20

dcaro created P13353 Command-Line Input.
Fri, Nov 20, 4:43 PM

Wed, Nov 18

dcaro updated the task description for T268140: Review the security groups on openstack.
Wed, Nov 18, 2:42 PM · cloud-services-team (Kanban)
dcaro created T268140: Review the security groups on openstack.
Wed, Nov 18, 2:37 PM · cloud-services-team (Kanban)

Mon, Nov 16

dcaro added a comment to T267864: Network flap on cloudbackup2002.

Isn't it better to put this one as 'depends on'? That way when we check for issues with the host cloudbackup2002 we will find an open task, that still will depend on the actual switch task, and that too allows us to track if the host is up again correctly after the switch is up no?

Mon, Nov 16, 10:55 AM · cloud-services-team (Hardware), Operations, ops-codfw

Tue, Nov 10

dcaro added a comment to T267433: Enable support for nested VMs.

As a workaround if needed and worst case scenario, you can try running bare/non-hardware backed qemu (really slow though :S)

Tue, Nov 10, 4:35 PM · cloud-services-team (Kanban), Cloud-VPS, Patch-For-Review
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Tue, Nov 10, 2:34 PM · Patch-For-Review, cloud-services-team (Kanban)
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Tue, Nov 10, 2:01 PM · Patch-For-Review, cloud-services-team (Kanban)
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Tue, Nov 10, 11:58 AM · Patch-For-Review, cloud-services-team (Kanban)
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Tue, Nov 10, 10:56 AM · Patch-For-Review, cloud-services-team (Kanban)
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Tue, Nov 10, 10:52 AM · Patch-For-Review, cloud-services-team (Kanban)
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Tue, Nov 10, 10:49 AM · Patch-For-Review, cloud-services-team (Kanban)

Fri, Nov 6

Bstorm awarded T267412: Provisioning/maintenance scripts a Yellow Medal token.
Fri, Nov 6, 3:38 PM · cloud-services-team (Kanban)
dcaro created T267412: Provisioning/maintenance scripts.
Fri, Nov 6, 3:36 PM · cloud-services-team (Kanban)
dcaro added a comment to T267369: Front proxy can keep bad routing info for webservices previously running on the grid engine.

A different fix would be to have the k8s backend send a message to the front proxy on process start telling the front proxy to clear any and all registered routes for the tool. This feels like a more robust solution.

Fri, Nov 6, 9:30 AM · cloud-services-team (Kanban), Toolforge

Thu, Nov 5

dcaro created P13231 (An Untitled Masterwork).
Thu, Nov 5, 3:59 PM
dcaro reassigned T267292: dcaro has same ssh key in wmcs and prod, prod ssh key revoked from dcaro to RobH.

Here is a new one:

Thu, Nov 5, 8:03 AM · Operations, SRE-Access-Requests

Wed, Nov 4

dcaro updated the task description for T267140: [toolsbeta] Rebuild servers to learn how to take down the services without downtime (and use affinities).
Wed, Nov 4, 12:12 PM · cloud-services-team (Kanban)

Tue, Nov 3

dcaro added a subtask for T267082: Rebuild Toolforge servers that should not have NFS mounted (and with affinity): T267140: [toolsbeta] Rebuild servers to learn how to take down the services without downtime (and use affinities).
Tue, Nov 3, 4:50 PM · cloud-services-team (Kanban)
dcaro added a parent task for T267140: [toolsbeta] Rebuild servers to learn how to take down the services without downtime (and use affinities): T267082: Rebuild Toolforge servers that should not have NFS mounted (and with affinity).
Tue, Nov 3, 4:50 PM · cloud-services-team (Kanban)
dcaro created T267140: [toolsbeta] Rebuild servers to learn how to take down the services without downtime (and use affinities).
Tue, Nov 3, 4:50 PM · cloud-services-team (Kanban)
dcaro created T267138: [toolsbeta] Make sure that /etc/security/access.conf is managed by puppet on all machines.
Tue, Nov 3, 4:05 PM · cloud-services-team (Kanban)
dcaro closed T267121: toolsbeta: Move all puppet config to toolsbeta-puppetmaster-04.toolsbeta.eqiad.wmflabs as Resolved.
Tue, Nov 3, 3:35 PM · cloud-services-team (Kanban)
dcaro triaged T267121: toolsbeta: Move all puppet config to toolsbeta-puppetmaster-04.toolsbeta.eqiad.wmflabs as Low priority.
Tue, Nov 3, 3:08 PM · cloud-services-team (Kanban)
dcaro added a comment to T267121: toolsbeta: Move all puppet config to toolsbeta-puppetmaster-04.toolsbeta.eqiad.wmflabs.

Note to myself, puppetdb02 hiera has puppetdb02 as master as overwrite, check and

Tue, Nov 3, 3:08 PM · cloud-services-team (Kanban)
dcaro claimed T267082: Rebuild Toolforge servers that should not have NFS mounted (and with affinity).
Tue, Nov 3, 1:53 PM · cloud-services-team (Kanban)
aborrero awarded T267121: toolsbeta: Move all puppet config to toolsbeta-puppetmaster-04.toolsbeta.eqiad.wmflabs a Cookie token.
Tue, Nov 3, 12:35 PM · cloud-services-team (Kanban)
dcaro created T267121: toolsbeta: Move all puppet config to toolsbeta-puppetmaster-04.toolsbeta.eqiad.wmflabs.
Tue, Nov 3, 11:58 AM · cloud-services-team (Kanban)

Mon, Nov 2

dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Mon, Nov 2, 4:34 PM · Patch-For-Review, cloud-services-team (Kanban)
dcaro created T267040: Requesting access to GLOBAL ROOT for David Caro.
Mon, Nov 2, 4:20 PM · SRE-Access-Requests, Operations
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Mon, Nov 2, 2:09 PM · Patch-For-Review, cloud-services-team (Kanban)
dcaro updated dcaro.
Mon, Nov 2, 1:44 PM
dcaro added a watcher for cloud-services-team: dcaro.
Mon, Nov 2, 1:42 PM
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Mon, Nov 2, 1:28 PM · Patch-For-Review, cloud-services-team (Kanban)
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Mon, Nov 2, 12:12 PM · Patch-For-Review, cloud-services-team (Kanban)
dcaro updated the task description for T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Mon, Nov 2, 12:11 PM · Patch-For-Review, cloud-services-team (Kanban)
dcaro claimed T266068: Onboard David Caro to Wikimedia Foundation as SRE in Cloud Services.
Mon, Nov 2, 12:02 PM · Patch-For-Review, cloud-services-team (Kanban)