Page MenuHomePhabricator

dancy (Ahmon Dancy)
Sr. Software Engineer, Release Engineering

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Jun 27 2020, 12:14 AM (110 w, 4 d)
Availability
Available
IRC Nick
dancy
LDAP User
Ahmon Dancy
MediaWiki User
ADancy (WMF) [ Global Accounts ]

Recent Activity

Mon, Aug 8

dancy updated the task description for T314776: Compressed AAAA returned by ruby resolv are different to those expected by ssh.
Mon, Aug 8, 2:42 PM · Patch-For-Review

Fri, Aug 5

dancy closed T308076: 1.39.0-wmf.23 deployment blockers as Resolved.
Fri, Aug 5, 3:16 PM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments

Thu, Aug 4

dancy updated the task description for T313972: Add gerrit2002 as a replica of gerrit1001.
Thu, Aug 4, 9:21 PM · Patch-For-Review, Gerrit, Release-Engineering-Team (The Decommission Mission 💀)
dancy added a comment to T313972: Add gerrit2002 as a replica of gerrit1001.

Confirming that replication from gerrit1001 to gerrit2002 is working.

Thu, Aug 4, 9:20 PM · Patch-For-Review, Gerrit, Release-Engineering-Team (The Decommission Mission 💀)

Wed, Aug 3

dancy closed T308074: 1.39.0-wmf.21 deployment blockers as Resolved.
Wed, Aug 3, 3:32 PM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments

Tue, Aug 2

dancy added a subtask for T308076: 1.39.0-wmf.23 deployment blockers: T314439: InvalidArgumentException: Wikimedia\Rdbms\JoinGroupBase::addJoin: $table must be either string, JoinGroup or SelectQueryBuilder.
Tue, Aug 2, 8:58 PM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
dancy added a parent task for T314439: InvalidArgumentException: Wikimedia\Rdbms\JoinGroupBase::addJoin: $table must be either string, JoinGroup or SelectQueryBuilder: T308076: 1.39.0-wmf.23 deployment blockers.
Tue, Aug 2, 8:58 PM · MW-1.39-notes (1.39.0-wmf.25; 2022-08-15), CirrusSearch, Discovery-Search, Wikimedia-production-error
dancy triaged T314439: InvalidArgumentException: Wikimedia\Rdbms\JoinGroupBase::addJoin: $table must be either string, JoinGroup or SelectQueryBuilder as Unbreak Now! priority.
Tue, Aug 2, 8:57 PM · MW-1.39-notes (1.39.0-wmf.25; 2022-08-15), CirrusSearch, Discovery-Search, Wikimedia-production-error
dancy updated the task description for T310395: Automated Tuesday Train via a timer.
Tue, Aug 2, 4:17 PM · Patch-For-Review, Scap, Release-Engineering-Team (Deployment Autopilot 🛩️)
dancy updated the task description for T310395: Automated Tuesday Train via a timer.
Tue, Aug 2, 4:00 PM · Patch-For-Review, Scap, Release-Engineering-Team (Deployment Autopilot 🛩️)

Mon, Aug 1

dancy reopened T302631: scap: add progress reporting to php-fpm-restarts as "Open".

Reopening since https://gerrit.wikimedia.org/r/793842 didn't quite do the trick.

Mon, Aug 1, 10:32 PM · Release-Engineering-Team (Next), Scap

Thu, Jul 21

dancy closed T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off), a subtask of T266055: Update Scap to perform rolling restart for all MW deploy, as Resolved.
Thu, Jul 21, 10:07 PM · Release-Engineering-Team (Radar), User-jijiki, Patch-For-Review, Scap
dancy closed T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off) as Resolved.
Thu, Jul 21, 10:07 PM · serviceops, Performance-Team, Scap
Dzahn awarded T313551: Add dancy to phabricator-roots a Like token.
Thu, Jul 21, 8:30 PM · SRE, Release-Engineering-Team (The Decommission Mission 💀), SRE-Access-Requests
dancy updated subscribers of T313551: Add dancy to phabricator-roots.
Thu, Jul 21, 8:27 PM · SRE, Release-Engineering-Team (The Decommission Mission 💀), SRE-Access-Requests
dancy updated subscribers of T313551: Add dancy to phabricator-roots.

Noting that my manager @thcipriani is on vacation right now.

Thu, Jul 21, 8:10 PM · SRE, Release-Engineering-Team (The Decommission Mission 💀), SRE-Access-Requests
dancy moved T313551: Add dancy to phabricator-roots from Backlog to Waiting for review on the Release-Engineering-Team (The Decommission Mission 💀) board.
Thu, Jul 21, 8:08 PM · SRE, Release-Engineering-Team (The Decommission Mission 💀), SRE-Access-Requests
dancy edited projects for T313551: Add dancy to phabricator-roots, added: Release-Engineering-Team (The Decommission Mission 💀); removed Release-Engineering-Team.
Thu, Jul 21, 8:08 PM · SRE, Release-Engineering-Team (The Decommission Mission 💀), SRE-Access-Requests
dancy created T313551: Add dancy to phabricator-roots.
Thu, Jul 21, 8:06 PM · SRE, Release-Engineering-Team (The Decommission Mission 💀), SRE-Access-Requests
dancy added a comment to T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).

I landed on a grace period of 3 minutes because the php-fpm-restart phase of scap currently takes about 2.5 minutes to complete.

Thu, Jul 21, 5:45 PM · serviceops, Performance-Team, Scap
dancy added a comment to T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).

If we want to preserve the same working logic as before, we'd have to increase the time delay (previously set to revalidate_freq) to cover the amount of time needed for FPM to have restarted itself. That's a messy thing to try to predict and setting it too high means that 1) you can't test things intuitively on mwdebug 2) you can't verify things intuitively in prod immediately and 3) we'd be running new opcache code after FPM restart but continue to use old possibly incompatible config.

Thu, Jul 21, 5:26 PM · serviceops, Performance-Team, Scap

Wed, Jul 20

dancy added a comment to T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).

Clarifying for the record: Opcache revalidation is turned off by setting opcache.validate_timestamps to 0. opcache.revalidate_freq still has a value of 10. So, the change I made in https://gerrit.wikimedia.org/r/815317 is effectively a non-change.

Wed, Jul 20, 9:15 PM · serviceops, Performance-Team, Scap
dancy reopened T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off) as "Open".

The same situation happened today during a deployment, so we're not out of the woods yet.

Wed, Jul 20, 8:33 PM · serviceops, Performance-Team, Scap
dancy reopened T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off), a subtask of T266055: Update Scap to perform rolling restart for all MW deploy, as Open.
Wed, Jul 20, 8:32 PM · Release-Engineering-Team (Radar), User-jijiki, Patch-For-Review, Scap
dancy renamed T313417: scap sync-file message during php-fpm-restarts about restart-php-fpm-all being called with an empty host list from scap sync-file message during php-fpm-restarts about restart-php-fmp-all being called with an empty host list to scap sync-file message during php-fpm-restarts about restart-php-fpm-all being called with an empty host list.
Wed, Jul 20, 4:09 PM · Scap
dancy closed T313417: scap sync-file message during php-fpm-restarts about restart-php-fpm-all being called with an empty host list as Resolved.

Scap 4.11.2 has been deployed with the fix.

Wed, Jul 20, 3:59 PM · Scap
dancy updated the task description for T313417: scap sync-file message during php-fpm-restarts about restart-php-fpm-all being called with an empty host list.
Wed, Jul 20, 3:14 PM · Scap
dancy claimed T313417: scap sync-file message during php-fpm-restarts about restart-php-fpm-all being called with an empty host list.

It is safe to ignore that message. I will work on eliminating it.

Wed, Jul 20, 3:13 PM · Scap

Tue, Jul 19

dancy updated the task description for T313250: Bring up gerrit2002.
Tue, Jul 19, 9:40 PM · Patch-For-Review, Gerrit, Release-Engineering-Team (The Decommission Mission 💀)
dancy renamed T313250: Bring up gerrit2002 from Decide how to bring up Gerrit2002 to Bring up Gerrit2002.
Tue, Jul 19, 9:39 PM · Patch-For-Review, Gerrit, Release-Engineering-Team (The Decommission Mission 💀)
dancy closed T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off), a subtask of T266055: Update Scap to perform rolling restart for all MW deploy, as Resolved.
Tue, Jul 19, 9:33 PM · Release-Engineering-Team (Radar), User-jijiki, Patch-For-Review, Scap
dancy closed T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off) as Resolved.
Tue, Jul 19, 9:33 PM · serviceops, Performance-Team, Scap
dancy closed T310242: Scap's php-fpm restart step "left" counter may be counter-intuitive as Resolved.
Tue, Jul 19, 7:13 PM · Release-Engineering-Team, Scap
dancy updated the task description for T313341: PHP Notice: Undefined property: Wikimedia\PSquare::$increments.
Tue, Jul 19, 6:33 PM · MW-1.39-notes (1.39.0-wmf.25; 2022-08-15), Performance-Team, RunningStat, Wikimedia-production-error
dancy updated the task description for T313341: PHP Notice: Undefined property: Wikimedia\PSquare::$increments.
Tue, Jul 19, 6:33 PM · MW-1.39-notes (1.39.0-wmf.25; 2022-08-15), Performance-Team, RunningStat, Wikimedia-production-error
dancy closed T310858: Avoid spammy scap warning when running on train-dev as Resolved.
Tue, Jul 19, 2:45 PM · Scap

Mon, Jul 18

dancy updated the task description for T313250: Bring up gerrit2002.
Mon, Jul 18, 7:21 PM · Patch-For-Review, Gerrit, Release-Engineering-Team (The Decommission Mission 💀)
dancy updated the task description for T243027: replacement for gerrit2001, decom gerrit2001.
Mon, Jul 18, 7:20 PM · Patch-For-Review, Release-Engineering-Team (The Decommission Mission 💀), serviceops-collab, serviceops, Gerrit, SRE
dancy updated subscribers of T310835: Scap pool counter error while backporting.
Mon, Jul 18, 5:01 PM · Release-Engineering-Team, Scap
dancy added a comment to T310835: Scap pool counter error while backporting.

I did three deployment sessions in the past week, each time encountered this during at least one of the syncs.

I hope this can be prioritised as we do not want deployers to get desensitised or otherwise normalize warning output during a Scap comment.

Mon, Jul 18, 4:36 PM · Release-Engineering-Team, Scap

Jul 1 2022

dancy added a comment to T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).

Thanks for the analysis Krinkle.

Jul 1 2022, 3:25 PM · serviceops, Performance-Team, Scap

Jun 30 2022

dancy triaged T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off) as High priority.

@Joe @Krinkle Jeena's evidence above shows that the php-fpm restart happened on the afflicted hosts. At this point I don't know what's going on but we're definitely in a state of decreased deployment confidence.

Jun 30 2022, 10:52 PM · serviceops, Performance-Team, Scap
dancy added a subtask for T266055: Update Scap to perform rolling restart for all MW deploy: T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).
Jun 30 2022, 10:47 PM · Release-Engineering-Team (Radar), User-jijiki, Patch-For-Review, Scap
dancy added a parent task for T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off): T266055: Update Scap to perform rolling restart for all MW deploy.
Jun 30 2022, 10:47 PM · serviceops, Performance-Team, Scap
dancy updated the task description for T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).
Jun 30 2022, 10:33 PM · serviceops, Performance-Team, Scap
dancy updated the task description for T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).
Jun 30 2022, 10:23 PM · serviceops, Performance-Team, Scap
dancy updated the task description for T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).
Jun 30 2022, 10:16 PM · serviceops, Performance-Team, Scap
dancy updated the task description for T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).
Jun 30 2022, 10:15 PM · serviceops, Performance-Team, Scap
dancy updated the title for P30698 A deployment which didn't take effect on all servers from untitled to A deployment which didn't take effect on all servers.
Jun 30 2022, 9:54 PM
dancy updated the task description for T311788: MW wmf-config tmp cache stays outdated after Scap deploy (opcache revalidation is off).
Jun 30 2022, 9:53 PM · serviceops, Performance-Team, Scap

Jun 29 2022

dancy closed T287046: scap backport --revert command, a subtask of T279322: Design m8s deployment workflows and tooling, as Resolved.
Jun 29 2022, 5:45 PM · User-brennen, Patch-For-Review, Release-Engineering-Team (Doing), MW-on-K8s, Release Pipeline
dancy closed T287046: scap backport --revert command as Resolved.
Jun 29 2022, 5:45 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), User-brennen, MW-on-K8s, Release Pipeline

Jun 28 2022

dancy moved T287046: scap backport --revert command from In progress to Done on the Release-Engineering-Team (Deployment Autopilot 🛩️) board.
Jun 28 2022, 10:06 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), User-brennen, MW-on-K8s, Release Pipeline
dancy closed T292646: Reduce latency of new Scap releases as Resolved.

Resolved via T303559

Jun 28 2022, 7:59 PM · serviceops, Release-Engineering-Team (Doing), SRE
dancy added a comment to T310395: Automated Tuesday Train via a timer.

Alternately, the branch cut job could write the branch name and/or version to an artifact (say MW_VERSION) and we could poll https://releases-jenkins.wikimedia.org/job/Automatic%20branch%20cut/lastSuccessfulBuild/artifact/MW_VERSION for changes

Jun 28 2022, 4:37 PM · Patch-For-Review, Scap, Release-Engineering-Team (Deployment Autopilot 🛩️)
dancy moved T303559: Use scap to deploy itself to scap targets from Done to In progress on the Release-Engineering-Team (Deployment Autopilot 🛩️) board.
Jun 28 2022, 3:37 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Patch-For-Review, Infrastructure-Foundations, serviceops, Scap
dancy added a comment to T310740: scap-o-scap: Bootstrapping a new host fails .

We need some way to ensure that the File['/usr/bin/scap'] resource (defined in operations/puppet/modules/scap/manifests/init.pp) is created before the scap3 provider (defined in operations/puppet/modules/scap/lib/puppet/provider/package/scap3.rb) is used.

Jun 28 2022, 3:22 PM · Patch-For-Review, Scap, Release-Engineering-Team
dancy closed T306425: Allow Scap to push to Gerrit without operator creds as Resolved.
Jun 28 2022, 3:06 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Scap
dancy closed T306425: Allow Scap to push to Gerrit without operator creds, a subtask of T310395: Automated Tuesday Train via a timer, as Resolved.
Jun 28 2022, 3:06 PM · Patch-For-Review, Scap, Release-Engineering-Team (Deployment Autopilot 🛩️)
dancy changed the status of T310740: scap-o-scap: Bootstrapping a new host fails from Stalled to In Progress.
Jun 28 2022, 2:23 PM · Patch-For-Review, Scap, Release-Engineering-Team
dancy changed the status of T310740: scap-o-scap: Bootstrapping a new host fails , a subtask of T303559: Use scap to deploy itself to scap targets, from Stalled to In Progress.
Jun 28 2022, 2:22 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Patch-For-Review, Infrastructure-Foundations, serviceops, Scap

Jun 27 2022

dancy added a comment to T294424: Jenkins search results missing /ci/ URL component.

This is fantastic. Thanks @hashar!

Jun 27 2022, 3:18 PM · Upstream, Continuous-Integration-Infrastructure, Jenkins

Jun 24 2022

dancy moved T287046: scap backport --revert command from Ready to In progress on the Release-Engineering-Team (Deployment Autopilot 🛩️) board.
Jun 24 2022, 9:57 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), User-brennen, MW-on-K8s, Release Pipeline
dancy added a comment to T310858: Avoid spammy scap warning when running on train-dev.

While something is worked out, I reverted the commit:
https://gerrit.wikimedia.org/r/c/mediawiki/tools/scap/+/808119

Jun 24 2022, 7:19 PM · Scap
dancy closed T309358: mw-on-k8s apache config missing cache-control for /static/ files as Resolved.

Re-opening for now. I closed it on the assumption it woud roll out more or less by itself within a few days, but it hasn't (either that, or the change didn't work).

To test, I'm using WikimediaDebug/k8s and with cache disabled comparing response to https://en.wikipedia.org/static/images/project-logos/enwiki.png with that of mwdebug1001, and no cache-control headers yet.

Jun 24 2022, 7:09 PM · Performance-Team (Radar), serviceops, MW-on-K8s
dancy moved T287045: Scap rollback command from Ready to Done on the Release-Engineering-Team (Deployment Autopilot 🛩️) board.
Jun 24 2022, 6:11 PM · Patch-For-Review, Release-Engineering-Team (Deployment Autopilot 🛩️), User-brennen, MW-on-K8s, Release Pipeline
dancy moved T306425: Allow Scap to push to Gerrit without operator creds from Ready to Done on the Release-Engineering-Team (Deployment Autopilot 🛩️) board.
Jun 24 2022, 6:00 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Scap
dancy moved T303559: Use scap to deploy itself to scap targets from In progress to Done on the Release-Engineering-Team (Deployment Autopilot 🛩️) board.
Jun 24 2022, 5:59 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Patch-For-Review, Infrastructure-Foundations, serviceops, Scap
dancy closed T308476: scap backport change_url: Deploy to mwdebug, a subtask of T287042: Scap backport change_url command, as Resolved.
Jun 24 2022, 5:59 PM · Release-Engineering-Team (Doing), User-brennen, MW-on-K8s, Release Pipeline
dancy closed T308476: scap backport change_url: Deploy to mwdebug as Resolved.
Jun 24 2022, 5:59 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), User-brennen, MW-on-K8s, Release Pipeline
dancy moved T308476: scap backport change_url: Deploy to mwdebug from Waiting for review to Done on the Release-Engineering-Team (Deployment Autopilot 🛩️) board.
Jun 24 2022, 5:59 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), User-brennen, MW-on-K8s, Release Pipeline

Jun 23 2022

dancy changed the status of T310740: scap-o-scap: Bootstrapping a new host fails , a subtask of T303559: Use scap to deploy itself to scap targets, from Open to Stalled.
Jun 23 2022, 9:26 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Patch-For-Review, Infrastructure-Foundations, serviceops, Scap
dancy changed the status of T310740: scap-o-scap: Bootstrapping a new host fails from Open to Stalled.

Lowering the priority of this ticket and keeping open for a bit until we have verification that new nodes are bootstrapped properly.

Jun 23 2022, 9:26 PM · Patch-For-Review, Scap, Release-Engineering-Team
dancy created T311240: Puppet agent failing on deploy-1004.devtools.eqiad1.wikimedia.cloud.
Jun 23 2022, 2:50 PM · cloud-services-team (Kanban), VPS-Projects

Jun 22 2022

dancy updated subscribers of T311182: scap error on 2022-06-21 deploy.

@Lucas_Werkmeister_WMDE also wrote:

I think there’s two issues there – the failed connection to lvs2010 (understandable if that was being worked on at the moment), and the fact that https://gerrit.wikimedia.org/g/operations/puppet/+/c8cb4a1796d5ff22803c171c277943eebecb8ee7/modules/conftool/files/safe-service-restart.py#333 throws an error if status was never assigned

Jun 22 2022, 9:39 PM · Scap
dancy added a project to T311181: Catalog of errors from scap (or subprocesses) related to php-fpm restart always being enabled: Release-Engineering-Team.
Jun 22 2022, 9:35 PM · Release-Engineering-Team, Scap
dancy added a subtask for T311181: Catalog of errors from scap (or subprocesses) related to php-fpm restart always being enabled: T311182: scap error on 2022-06-21 deploy.
Jun 22 2022, 9:34 PM · Release-Engineering-Team, Scap
dancy added a parent task for T311182: scap error on 2022-06-21 deploy: T311181: Catalog of errors from scap (or subprocesses) related to php-fpm restart always being enabled.
Jun 22 2022, 9:34 PM · Scap
dancy created T311182: scap error on 2022-06-21 deploy.
Jun 22 2022, 9:34 PM · Scap
dancy added a subtask for T311181: Catalog of errors from scap (or subprocesses) related to php-fpm restart always being enabled: T310835: Scap pool counter error while backporting.
Jun 22 2022, 9:30 PM · Release-Engineering-Team, Scap
dancy added a parent task for T310835: Scap pool counter error while backporting: T311181: Catalog of errors from scap (or subprocesses) related to php-fpm restart always being enabled.
Jun 22 2022, 9:30 PM · Release-Engineering-Team, Scap
dancy created T311181: Catalog of errors from scap (or subprocesses) related to php-fpm restart always being enabled.
Jun 22 2022, 9:30 PM · Release-Engineering-Team, Scap
dancy added a comment to T310620: Requesting SSH keypair for deployment server keyholder to push to Gerrit.

I added the public key to Gerrit's trainbranchbot using the following command:

echo ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIA9PnDpx0+F5mgJUbLxiCOFm2G5anUPaZJvgG9d+XhMk /etc/keyholder.d/trainbranchbot | ssh \
-p29418 gerrit.wikimedia.org gerrit set-account trainbranchbot --add-ssh-key -
Jun 22 2022, 5:39 PM · serviceops, SRE
dancy added a comment to T310620: Requesting SSH keypair for deployment server keyholder to push to Gerrit.

Thank you @akosiaris !

Jun 22 2022, 4:56 PM · serviceops, SRE
dancy assigned T310395: Automated Tuesday Train via a timer to hashar.
Jun 22 2022, 4:11 PM · Patch-For-Review, Scap, Release-Engineering-Team (Deployment Autopilot 🛩️)
dancy added a comment to T266055: Update Scap to perform rolling restart for all MW deploy.

Hey @dancy, do you think we're good to proceed with disabling revalidation for all of api,app,parsoid ?

Jun 22 2022, 3:54 PM · Release-Engineering-Team (Radar), User-jijiki, Patch-For-Review, Scap
dancy added a comment to T310740: scap-o-scap: Bootstrapping a new host fails .

Sorry for the delay in commenting.

That patch currently applies the new bootstrapping config in the scap class, but it needs access to at least one of the different hieradata values declaring deployment servers/masters.

Is this still required? or is it resolved with the exec constraint mentioned by dancy? if still required could someone point me to how/where it is to be used?

Jun 22 2022, 3:29 PM · Patch-For-Review, Scap, Release-Engineering-Team

Jun 21 2022

dancy added a comment to T310740: scap-o-scap: Bootstrapping a new host fails .

@jnuche I landed on making the bootstrap-scap-target exec resource only run if ${scap_home}/scap/bin/scap does not exist. That will prevent it from running on already-bootstrapped nodes, which is what we want anyway.

Jun 21 2022, 9:46 PM · Patch-For-Review, Scap, Release-Engineering-Team
dancy updated the task description for T303559: Use scap to deploy itself to scap targets.
Jun 21 2022, 9:31 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Patch-For-Review, Infrastructure-Foundations, serviceops, Scap
dancy updated the task description for T303559: Use scap to deploy itself to scap targets.
Jun 21 2022, 9:29 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Patch-For-Review, Infrastructure-Foundations, serviceops, Scap
dancy updated the task description for T303559: Use scap to deploy itself to scap targets.
Jun 21 2022, 5:42 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Patch-For-Review, Infrastructure-Foundations, serviceops, Scap
dancy updated the task description for T303559: Use scap to deploy itself to scap targets.
Jun 21 2022, 5:41 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Patch-For-Review, Infrastructure-Foundations, serviceops, Scap
dancy added a project to T311080: All scap error messages should include the version of scap that is running: Scap.
Jun 21 2022, 4:41 PM · Release-Engineering-Team (Onboarding 🚀), Scap
dancy created T311080: All scap error messages should include the version of scap that is running.
Jun 21 2022, 4:41 PM · Release-Engineering-Team (Onboarding 🚀), Scap
dancy updated subscribers of T310620: Requesting SSH keypair for deployment server keyholder to push to Gerrit.

Pinging @JMeybohm and @Dzahn for support.

Jun 21 2022, 4:14 PM · serviceops, SRE
dancy updated the task description for T310395: Automated Tuesday Train via a timer.
Jun 21 2022, 3:56 PM · Patch-For-Review, Scap, Release-Engineering-Team (Deployment Autopilot 🛩️)
dancy updated the task description for T310858: Avoid spammy scap warning when running on train-dev.
Jun 21 2022, 2:53 PM · Scap

Jun 16 2022

dancy added a comment to T308271: Deploy buildkitd to trusted GitLab runners.

Thank you very much @Dzahn !

Jun 16 2022, 10:00 PM · Patch-For-Review, User-brennen, GitLab (CI & Job Runners), Release-Engineering-Team (GitLab-a-thon 🦊)
dancy closed T310699: Scap sync-file fails with error about pause_after_testserver_sync as Resolved.
Jun 16 2022, 8:38 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Deployments
dancy added a subtask for T303559: Use scap to deploy itself to scap targets: T310740: scap-o-scap: Bootstrapping a new host fails .
Jun 16 2022, 8:36 PM · Release-Engineering-Team (Deployment Autopilot 🛩️), Patch-For-Review, Infrastructure-Foundations, serviceops, Scap