Page MenuHomePhabricator

dcaro (David Caro)
SRE & amauteur yak shaver

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Monday

  • Clear sailing ahead.

User Details

User Since
Nov 2 2020, 11:59 AM (99 w, 4 d)
Availability
Available
IRC Nick
dcaro
LDAP User
David Caro
MediaWiki User
DCaro (WMF) [ Global Accounts ]

Recent Activity

Yesterday

nskaggs awarded T318653: [alerting] Use the `services` label in cookbook to silence groups of alerts a 100 token.
Fri, Sep 30, 2:01 PM · Cloud-Services-Origin-Team, Cloud-Services-Worktype-Project, User-dcaro, cloud-services-team (Kanban)
dcaro updated subscribers of T319001: Degraded RAID on cloudvirt1023.

@Andrew fyi

Fri, Sep 30, 8:34 AM · cloud-services-team (Kanban), SRE, ops-eqiad

Wed, Sep 28

dcaro added a comment to T315593: clean up stray VM backup snapshots.

Looks good! :)

Wed, Sep 28, 4:19 PM · cloud-services-team (Kanban)
dcaro added a project to T318816: Test, understand, improve rabbitmq HA: User-dcaro.
Wed, Sep 28, 2:36 PM · User-dcaro, cloud-services-team (Kanban)

Tue, Sep 27

dcaro added a project to T317651: hw troubleshooting: network cards shutting down for lasbtore1005.eqiad.wmnet: ops-eqiad.
Tue, Sep 27, 4:45 PM · SRE, ops-eqiad, cloud-services-team (Kanban), DC-Ops
dcaro added a comment to T318680: Ceph cookbook fails when checking jumbo frames.

I did the same before too, the issue is sometimes hard to reproduce.

Tue, Sep 27, 11:57 AM · Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban)
dcaro added a comment to T316866: Decision Request - Openstack Upgrade Cadence.

In Option 4:

Tue, Sep 27, 9:49 AM · Cloud Services Proposals
dcaro added a comment to T316866: Decision Request - Openstack Upgrade Cadence.

No goals met

Tue, Sep 27, 9:41 AM · Cloud Services Proposals
dcaro assigned T317576: SystemdUnitDownForLong cloudcontrol1005:9100 Unit keystone_sync_keys_to_cloudcontrol1006.wikimedia.org.service on node cloudcontrol1005 has been down for long. to Andrew.
Tue, Sep 27, 9:29 AM · cloud-services-team (Kanban)
dcaro assigned T316390: SystemdUnitDownForLong cloudcontrol1005:9100 Unit backup_cinder_volumes.service on node cloudcontrol1005 has been down for long. to Andrew.
Tue, Sep 27, 9:28 AM · cloud-services-team (Kanban)
dcaro reassigned T316738: SystemdUnitDownForLong cloudcontrol1005:9100 Unit purge_vm_backup.service on node cloudcontrol1005 has been down for long. from dcaro to Andrew.
Tue, Sep 27, 9:28 AM · Patch-For-Review, cloud-services-team (Kanban)
dcaro closed T318483: NeutronAgentDown openstack.eqiad1.wikimediacloud.org:12345 A Neutron agent is down, VMs will have connectivity issues as Resolved.

Controlled action

Tue, Sep 27, 9:24 AM · cloud-services-team (Kanban)
dcaro closed T318499: NodeDownForLong cloudvirt1053:9100 Node cloudvirt1053 has been down for long. as Resolved.

Controlled action

Tue, Sep 27, 9:24 AM · cloud-services-team (Kanban)
dcaro closed T318497: NodeDown as Resolved.

Controlled action

Tue, Sep 27, 9:23 AM · cloud-services-team (Kanban)
dcaro moved T316657: [tbs.devenv] Raymond to setup the development environment from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:35 AM · Toolforge Build Service (Iteration 01), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro moved T316335: [tbs.registry-admission-webhook] Ensure we use the deploy.sh standard from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:34 AM · Toolforge Build Service (Iteration 01), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro closed T316335: [tbs.registry-admission-webhook] Ensure we use the deploy.sh standard, a subtask of T267374: [tbs.beta] Create a toolforge build service beta release, as Resolved.
Tue, Sep 27, 8:34 AM · Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, Toolforge Build Service, User-dcaro, cloud-services-team (Kanban)
dcaro closed T316335: [tbs.registry-admission-webhook] Ensure we use the deploy.sh standard as Resolved.
Tue, Sep 27, 8:34 AM · Toolforge Build Service (Iteration 01), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro closed T316326: [tbs.buildservice] Ensure we use the deploy.sh standard as Resolved.
Tue, Sep 27, 8:33 AM · Patch-For-Review, Toolforge Build Service (Iteration 00), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro closed T316326: [tbs.buildservice] Ensure we use the deploy.sh standard, a subtask of T316325: [tbs.buildservice.toolsbeta] Deploy on toolsbeta, as Resolved.
Tue, Sep 27, 8:33 AM · Toolforge Build Service, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro moved T316326: [tbs.buildservice] Ensure we use the deploy.sh standard from In Review to Done on the Toolforge Build Service (Iteration 00) board.
Tue, Sep 27, 8:33 AM · Patch-For-Review, Toolforge Build Service (Iteration 00), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro closed T316222: [tbs.cli]: Create reasonable defaults for the build command (so the users have to add only the minimum needed info), a subtask of T267374: [tbs.beta] Create a toolforge build service beta release, as Resolved.
Tue, Sep 27, 8:32 AM · Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, Toolforge Build Service, User-dcaro, cloud-services-team (Kanban)
dcaro closed T316222: [tbs.cli]: Create reasonable defaults for the build command (so the users have to add only the minimum needed info) as Resolved.
Tue, Sep 27, 8:32 AM · Toolforge Build Service (Iteration 00), User-Slst2020, User-dcaro, cloud-services-team (Kanban)
dcaro moved T316226: [tbs.cli]: toolforge build-show shows unknown for steps that are actually ok from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:32 AM · Toolforge Build Service (Iteration 01), User-Slst2020, User-dcaro, Cloud-Services-Origin-Team, Cloud-Services-Worktype-Project, cloud-services-team (Kanban)
dcaro moved T316339: [tbs.registry-admission.tools] Allow images from harbor on tools from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:32 AM · Toolforge Build Service (Iteration 01), Patch-For-Review, User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, User-dcaro, cloud-services-team (Kanban)
dcaro moved T316332: [tbs.buildpack-admission-controller] Ensure we use the deploy.sh standard from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:27 AM · Toolforge Build Service (Iteration 01), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro moved T316232: [tbs.harbor.toolsbeta] Create a postgres DB with trove and plug the current instance there from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:26 AM · Toolforge Build Service (Iteration 01), User-Slst2020, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro moved T316656: [tbs.devenv] Slavina to setup the development environment from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:26 AM · Toolforge Build Service (Iteration 01), User-Slst2020, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro moved T316658: [tbs.devenv] Francesco to setup the development environment from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:25 AM · Toolforge Build Service (Iteration 01), Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro moved T316228: [tbs.cli] Define the process to build and deploy the CLI on toolforge bastions from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:25 AM · Toolforge Build Service (Iteration 01), Patch-For-Review, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro moved T316854: [tbs.buildpacks] The uwsgi buildpack is failing to run due to missing Python.h library from Iteration 00 to Iteration 01 on the Toolforge Build Service board.
Tue, Sep 27, 8:24 AM · Toolforge Build Service (Iteration 01), Patch-For-Review, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro closed T316328: [tbs.registry-admission.toolsbeta] Allow images from harbor on toolsbeta, a subtask of T316330: [tbs.toolsbeta] Deploy a tool using a buildpack-based image on Toolsbeta, as Resolved.
Tue, Sep 27, 8:23 AM · Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro closed T316328: [tbs.registry-admission.toolsbeta] Allow images from harbor on toolsbeta as Resolved.
Tue, Sep 27, 8:23 AM · Toolforge Build Service (Iteration 00), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro reopened T316328: [tbs.registry-admission.toolsbeta] Allow images from harbor on toolsbeta as "Open".
Tue, Sep 27, 8:22 AM · Toolforge Build Service (Iteration 00), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro reopened T316328: [tbs.registry-admission.toolsbeta] Allow images from harbor on toolsbeta, a subtask of T316330: [tbs.toolsbeta] Deploy a tool using a buildpack-based image on Toolsbeta, as Open.
Tue, Sep 27, 8:22 AM · Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro closed T316328: [tbs.registry-admission.toolsbeta] Allow images from harbor on toolsbeta, a subtask of T316330: [tbs.toolsbeta] Deploy a tool using a buildpack-based image on Toolsbeta, as Resolved.
Tue, Sep 27, 8:22 AM · Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro closed T316328: [tbs.registry-admission.toolsbeta] Allow images from harbor on toolsbeta as Resolved.
Tue, Sep 27, 8:22 AM · Toolforge Build Service (Iteration 00), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro added a project to T318656: [tbs][cli] Error when there's no 'steps' in the status: Toolforge Build Service.
Tue, Sep 27, 8:21 AM · Toolforge Build Service, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro added a subtask for T267374: [tbs.beta] Create a toolforge build service beta release: T318656: [tbs][cli] Error when there's no 'steps' in the status.
Tue, Sep 27, 8:21 AM · Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, Toolforge Build Service, User-dcaro, cloud-services-team (Kanban)
dcaro added a parent task for T318656: [tbs][cli] Error when there's no 'steps' in the status: T267374: [tbs.beta] Create a toolforge build service beta release.
Tue, Sep 27, 8:21 AM · Toolforge Build Service, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro triaged T318656: [tbs][cli] Error when there's no 'steps' in the status as High priority.
Tue, Sep 27, 8:21 AM · Toolforge Build Service, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro created T318654: [alerting][cookbooks] When removing silences, give some time for the alerts to clear.
Tue, Sep 27, 8:01 AM · Cloud-Services-Origin-Team, Cloud-Services-Worktype-Project, User-dcaro, cloud-services-team (Kanban)
dcaro created T318653: [alerting] Use the `services` label in cookbook to silence groups of alerts.
Tue, Sep 27, 7:59 AM · Cloud-Services-Origin-Team, Cloud-Services-Worktype-Project, User-dcaro, cloud-services-team (Kanban)
dcaro created T318650: [alerting] Make sure the 'instance' label is populated with the 'hostname' label for openstack metrics.
Tue, Sep 27, 7:50 AM · Cloud-Services-Origin-Team, Cloud-Services-Worktype-Project, User-dcaro, cloud-services-team (Kanban)
Enterprisey awarded T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases a Barnstar token.
Tue, Sep 27, 12:00 AM · User-dcaro, Data-Services, cloud-services-team (Kanban)

Mon, Sep 26

dcaro moved T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases from Doing to Done on the User-dcaro board.
Mon, Sep 26, 12:48 PM · User-dcaro, Data-Services, cloud-services-team (Kanban)
dcaro closed T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases as Resolved.
Mon, Sep 26, 12:48 PM · User-dcaro, Data-Services, cloud-services-team (Kanban)
dcaro added a comment to T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases.

Now the account was created :)

Sep 26 12:39:47 labstore1004 /usr/local/sbin/maintain-dbusers[63140]: Created account in clouddb1015.eqiad.wmnet:3314 for paws Inaki LL
Mon, Sep 26, 12:48 PM · User-dcaro, Data-Services, cloud-services-team (Kanban)
dcaro claimed T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases.
Mon, Sep 26, 12:35 PM · User-dcaro, Data-Services, cloud-services-team (Kanban)
dcaro moved T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases from To refine to Doing on the User-dcaro board.
Mon, Sep 26, 12:35 PM · User-dcaro, Data-Services, cloud-services-team (Kanban)
dcaro changed the status of T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases from Open to In Progress.
Mon, Sep 26, 12:35 PM · User-dcaro, Data-Services, cloud-services-team (Kanban)
dcaro added a project to T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases: User-dcaro.
Mon, Sep 26, 12:35 PM · User-dcaro, Data-Services, cloud-services-team (Kanban)
dcaro added a comment to T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases.

Testing locally, using a non-case insensitive collation (one not ending in _ci) seems to do the trick, used utf8mb4_bin:

MariaDB [test1]> CREATE TABLE `account` (   `id` int(10) unsigned NOT NULL AUTO_INCREMENT,   `mysql_username` varchar(255) NOT NULL,   `type` enum('user','tool','paws') DEFAULT NULL,   `username` varchar(255) NOT NULL,   `password_hash` binary(41) NOT NULL,   PRIMARY KEY (`id`),   UNIQUE KEY `account_type` (`type`,`username`) ) ENGINE=InnoDB AUTO_INCREMENT=28887 DEFAULT CHARSET=utf8mb4 ROW_FORMAT=DYNAMIC;
Query OK, 0 rows affected (0.073 sec)
Mon, Sep 26, 8:38 AM · User-dcaro, Data-Services, cloud-services-team (Kanban)
dcaro added a comment to T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases.

It seems that it's trying to add a new entry for user 'Inaki LL' and when updating the index it collides with an existing 'Iñaki LL', looking if this might be a collation issue (!= charset, see https://mariadb.com/kb/en/setting-character-sets-and-collations/).

Mon, Sep 26, 8:08 AM · User-dcaro, Data-Services, cloud-services-team (Kanban)

Fri, Sep 16

dcaro updated subscribers of T316436: Cloud VPS Terraform support.
Fri, Sep 16, 2:23 PM · Epic, Cloud-VPS, cloud-services-team (Kanban)

Thu, Sep 15

dcaro assigned T317838: SystemdUnitDownForLong cloudcontrol1006:9100 Unit keystone_sync_keys_to_cloudcontrol1005.wikimedia.org.service on node cloudcontrol1006 has been down for long. to Andrew.

I'm not sure if you are still thinking about solving this, we can continue trying to fix it or if not an issue we should try to avoid the alert.

Thu, Sep 15, 7:10 AM · Patch-For-Review, cloud-services-team (Kanban)

Wed, Sep 14

dcaro added a comment to T316328: [tbs.registry-admission.toolsbeta] Allow images from harbor on toolsbeta.

Yep, it does :), so that part is done, the next one is deploying it on toolsbeta.

Wed, Sep 14, 5:05 PM · Toolforge Build Service (Iteration 00), User-Raymond_Ndibe, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro added a comment to T317643: 2022-09-13 labstore1005 down.

btw. backups are also failing as they pull from the secondary (labstore1005)

Wed, Sep 14, 9:04 AM · Patch-For-Review, Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (Kanban), User-dcaro

Tue, Sep 13

dcaro added a parent task for T317651: hw troubleshooting: network cards shutting down for lasbtore1005.eqiad.wmnet: T317643: 2022-09-13 labstore1005 down.
Tue, Sep 13, 2:53 PM · SRE, ops-eqiad, cloud-services-team (Kanban), DC-Ops
dcaro added a subtask for T317643: 2022-09-13 labstore1005 down: T317651: hw troubleshooting: network cards shutting down for lasbtore1005.eqiad.wmnet.
Tue, Sep 13, 2:53 PM · Patch-For-Review, Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (Kanban), User-dcaro
dcaro created T317651: hw troubleshooting: network cards shutting down for lasbtore1005.eqiad.wmnet.
Tue, Sep 13, 2:52 PM · SRE, ops-eqiad, cloud-services-team (Kanban), DC-Ops
dcaro added a comment to T317643: 2022-09-13 labstore1005 down.

It seems that the network cards on labstore1005 overheated and decided to turn themselves off:

[10704668.955975] bnx2x: [bnx2x_attn_int_deasserted0:4170(enp133s0f0)]SPIO5 hw attention
[10704668.964747] bnx2x 0000:85:00.0 enp133s0f0: Fan Failure on Network Controller has caused the driver to shutdown the card to prevent permanent damage.
                  Please contact OEM Support for assistance
[10704669.040066] bnx2x: [bnx2x_attn_int_deasserted0:4170(enp133s0f1)]SPIO5 hw attention
[10704669.048824] bnx2x 0000:85:00.1 enp133s0f1: Fan Failure on Network Controller has caused the driver to shutdown the card to prevent permanent damage.
                  Please contact OEM Support for assistance
Tue, Sep 13, 2:25 PM · Patch-For-Review, Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (Kanban), User-dcaro
dcaro changed the status of T317643: 2022-09-13 labstore1005 down from Open to In Progress.
Tue, Sep 13, 2:17 PM · Patch-For-Review, Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (Kanban), User-dcaro
dcaro moved T317643: 2022-09-13 labstore1005 down from To refine to Doing on the User-dcaro board.
Tue, Sep 13, 2:17 PM · Patch-For-Review, Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (Kanban), User-dcaro
dcaro triaged T317643: 2022-09-13 labstore1005 down as High priority.
Tue, Sep 13, 2:17 PM · Patch-For-Review, Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (Kanban), User-dcaro
dcaro merged T317571: SystemdUnitDownForLong labstore1007:9100 Unit ferm.service on node labstore1007 has been down for long. into T317596: SystemdUnitDownForLong labstore1007:9100 - ferm.service.
Tue, Sep 13, 7:46 AM · cloud-services-team (Kanban)
dcaro merged task T317571: SystemdUnitDownForLong labstore1007:9100 Unit ferm.service on node labstore1007 has been down for long. into T317596: SystemdUnitDownForLong labstore1007:9100 - ferm.service.
Tue, Sep 13, 7:46 AM · cloud-services-team (Kanban)
dcaro renamed T317596: SystemdUnitDownForLong labstore1007:9100 - ferm.service from SystemdUnitDownForLong labstore1007:9100 to SystemdUnitDownForLong labstore1007:9100 - ferm.service.
Tue, Sep 13, 7:27 AM · cloud-services-team (Kanban)
dcaro closed T317596: SystemdUnitDownForLong labstore1007:9100 - ferm.service as Resolved.

This actually was not related (I think).

Tue, Sep 13, 7:27 AM · cloud-services-team (Kanban)
dcaro assigned T317593: SystemdUnitDownForLong clouddumps1002:9100 Unit kiwix-mirror-update.service on node clouddumps1002 has been down for long. to Andrew.
Tue, Sep 13, 7:19 AM · cloud-services-team (Kanban)
dcaro assigned T317596: SystemdUnitDownForLong labstore1007:9100 - ferm.service to Andrew.
Tue, Sep 13, 7:19 AM · cloud-services-team (Kanban)
dcaro assigned T317597: SystemdUnitDownForLong clouddumps1001:9100 Unit kiwix-mirror-update.service on node clouddumps1001 has been down for long. to Andrew.

@Andrew I'm not sure if you are still working on these, let me know otherwise.

Tue, Sep 13, 7:19 AM · cloud-services-team (Kanban)

Fri, Sep 9

dcaro changed the status of T316228: [tbs.cli] Define the process to build and deploy the CLI on toolforge bastions from In Progress to Stalled.
Fri, Sep 9, 1:31 PM · Toolforge Build Service (Iteration 01), Patch-For-Review, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro moved T316228: [tbs.cli] Define the process to build and deploy the CLI on toolforge bastions from Doing to Blocked on the User-dcaro board.
Fri, Sep 9, 1:31 PM · Toolforge Build Service (Iteration 01), Patch-For-Review, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro changed the status of T316228: [tbs.cli] Define the process to build and deploy the CLI on toolforge bastions, a subtask of T267374: [tbs.beta] Create a toolforge build service beta release, from In Progress to Stalled.
Fri, Sep 9, 1:31 PM · Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, Toolforge Build Service, User-dcaro, cloud-services-team (Kanban)
dcaro moved T316854: [tbs.buildpacks] The uwsgi buildpack is failing to run due to missing Python.h library from Doing to Blocked on the User-dcaro board.
Fri, Sep 9, 1:30 PM · Toolforge Build Service (Iteration 01), Patch-For-Review, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro changed the status of T316854: [tbs.buildpacks] The uwsgi buildpack is failing to run due to missing Python.h library from In Progress to Stalled.
Fri, Sep 9, 1:30 PM · Toolforge Build Service (Iteration 01), Patch-For-Review, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro closed T316919: [novafullstack.alerts] stop paging and instead create tasks for any nova-fullstack related alert as Resolved.
Fri, Sep 9, 1:15 PM · Patch-For-Review, Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro moved T316919: [novafullstack.alerts] stop paging and instead create tasks for any nova-fullstack related alert from Doing to Done on the User-dcaro board.
Fri, Sep 9, 1:15 PM · Patch-For-Review, Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro

Wed, Sep 7

dcaro added a comment to T316222: [tbs.cli]: Create reasonable defaults for the build command (so the users have to add only the minimum needed info).

Done, you should have access

Wed, Sep 7, 4:02 PM · Toolforge Build Service (Iteration 00), User-Slst2020, User-dcaro, cloud-services-team (Kanban)
dcaro added a comment to T316222: [tbs.cli]: Create reasonable defaults for the build command (so the users have to add only the minimum needed info).

Oh! let's fix that then!

Wed, Sep 7, 4:00 PM · Toolforge Build Service (Iteration 00), User-Slst2020, User-dcaro, cloud-services-team (Kanban)
dcaro moved T316854: [tbs.buildpacks] The uwsgi buildpack is failing to run due to missing Python.h library from In Progress to In Review on the Toolforge Build Service (Iteration 00) board.
Wed, Sep 7, 3:55 PM · Toolforge Build Service (Iteration 01), Patch-For-Review, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro added a comment to T316222: [tbs.cli]: Create reasonable defaults for the build command (so the users have to add only the minimum needed info).

Nothing, the patch just needs merging if it looks good to everyone.

Wed, Sep 7, 3:47 PM · Toolforge Build Service (Iteration 00), User-Slst2020, User-dcaro, cloud-services-team (Kanban)
dcaro added a comment to T316232: [tbs.harbor.toolsbeta] Create a postgres DB with trove and plug the current instance there.

Document also where it needs to be configured in the README for the buildservice repository:
https://github.com/toolforge/buildservice

@dcaro do you mean how to configure the docker-compose setup to work with the external db?

Wed, Sep 7, 2:38 PM · Toolforge Build Service (Iteration 01), User-Slst2020, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro added a comment to T316222: [tbs.cli]: Create reasonable defaults for the build command (so the users have to add only the minimum needed info).

What is left here? What is blocking you?

Wed, Sep 7, 12:39 PM · Toolforge Build Service (Iteration 00), User-Slst2020, User-dcaro, cloud-services-team (Kanban)
dcaro added a comment to T317143: [tbs] Update the tekton cli package.

Just created https://gitlab.wikimedia.org/repos/cloud/toolforge/packaging-pack/-/tree/main/ to package the pack cli, the tekton cli is just a binary too so you might be able to reuse all the scripts there (it was surprisingly complicated :/)

Wed, Sep 7, 10:17 AM · Toolforge Build Service, Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro

Tue, Sep 6

dcaro added a subtask for T265684: Figure out CI solution for building buildpack-based images for Toolforge: T317143: [tbs] Update the tekton cli package.
Tue, Sep 6, 4:44 PM · cloud-services-team (Kanban), Toolforge
dcaro added a parent task for T317143: [tbs] Update the tekton cli package: T265684: Figure out CI solution for building buildpack-based images for Toolforge.
Tue, Sep 6, 4:44 PM · Toolforge Build Service, Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro removed a project from T317143: [tbs] Update the tekton cli package: Toolforge Build Service (Iteration 00).
Tue, Sep 6, 4:42 PM · Toolforge Build Service, Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro edited projects for T317143: [tbs] Update the tekton cli package, added: Toolforge Build Service (Iteration 00); removed Toolforge Build Service.
Tue, Sep 6, 4:42 PM · Toolforge Build Service, Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro added a project to T317143: [tbs] Update the tekton cli package: Toolforge Build Service.
Tue, Sep 6, 4:42 PM · Toolforge Build Service, Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro triaged T317143: [tbs] Update the tekton cli package as Medium priority.
Tue, Sep 6, 4:42 PM · Toolforge Build Service, Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
dcaro assigned T316984: SystemdUnitDownForLong cloudbackup1003:9100 Unit backup_glance_images.service on node cloudbackup1003 has been down for long. to Andrew.
Tue, Sep 6, 9:56 AM · cloud-services-team (Kanban)
dcaro closed T316989: SystemdUnitDownForLong cloudcontrol1005:9100 Unit keystone_sync_keys_to_cloudcontrol1007.wikimedia.org.service on node cloudcontrol1005 has been down for long. as Resolved.

Closing as it's solved, but pinging @Andrew as he was playing with this at some point.

Tue, Sep 6, 8:30 AM · cloud-services-team (Kanban)
dcaro committed rODIT8cc49512ba23: build: use the standard path to get the docker binary (authored by dcaro).
build: use the standard path to get the docker binary
Tue, Sep 6, 8:22 AM
dcaro committed rODITd9cf23a220ae: bullseye0: Improve the install-packages script (authored by dcaro).
bullseye0: Improve the install-packages script
Tue, Sep 6, 8:22 AM
dcaro committed rODIT801ea3f7cfe4: Remove buster0 buildpacks images (authored by dcaro).
Remove buster0 buildpacks images
Tue, Sep 6, 8:22 AM
dcaro committed rODIT37a95b92db31: bullseye0: Add bullseye buildpack build/run images (authored by dcaro).
bullseye0: Add bullseye buildpack build/run images
Tue, Sep 6, 8:22 AM
dcaro moved T317071: PuppetSyncFailure on project-proxy, toolsbeta and cloudinfra cloudvps projects from Doing to Blocked on the User-dcaro board.
Tue, Sep 6, 7:43 AM · Patch-For-Review, Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (Kanban), User-dcaro
dcaro changed the status of T317071: PuppetSyncFailure on project-proxy, toolsbeta and cloudinfra cloudvps projects from In Progress to Stalled.
Tue, Sep 6, 7:43 AM · Patch-For-Review, Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (Kanban), User-dcaro