Page MenuHomePhabricator

aborrero (arturo)
SRE at Wikimedia Cloud Services Team

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Oct 23 2017, 12:19 PM (334 w, 15 h)
Availability
Available
IRC Nick
arturo
LDAP User
Arturo Borrero Gonzalez
MediaWiki User
ABorrero (WMF) [ Global Accounts ]

I'm Arturo Borrero Gonzalez from Spain (Seville). I'm Site Reliability Engineer (SRE) in the Wikimedia Cloud Services Team, a Wikimedia Foundation staff.

You may find me in some FLOSS projects, like Netfilter and Debian.

Recent Activity

Yesterday

aborrero added a comment to T335131: Toolforge: replace admission controllers with an existing policy admin project.

Just noticed:

Mon, Mar 18, 1:50 PM · User-aborrero, Kubernetes, Toolforge
aborrero closed T307651: Upgrade Toolforge Kubernetes to version 1.24 as Resolved.

completed.

Mon, Mar 18, 1:35 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero closed T307651: Upgrade Toolforge Kubernetes to version 1.24, a subtask of T316107: Upgrade Toolforge Kubernetes to version 1.25, as Resolved.
Mon, Mar 18, 1:33 PM · cloud-services-team, Toolforge
aborrero added a comment to T307651: Upgrade Toolforge Kubernetes to version 1.24.

the control plane is now upgraded:

Mon, Mar 18, 11:27 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero committed rCCKB2472fcd2c65d: toolforge.k8s.worker.upgrade: give a hint about static pod restarts (authored by aborrero).
toolforge.k8s.worker.upgrade: give a hint about static pod restarts
Mon, Mar 18, 11:25 AM
aborrero added a comment to T307651: Upgrade Toolforge Kubernetes to version 1.24.

saving this info here in case is required later:

----- OUTPUT of 'sudo -i kubeadm ...ade plan 1.24.17' -----                                                                                  
[upgrade/config] Making sure the configuration is correct:                                                                                   
[upgrade/config] Reading configuration from the cluster...                                                                                   
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'                           
W0318 10:53:57.579543 1176002 initconfiguration.go:120] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/run/containerd/containerd.sock". Please update your configuration!
[preflight] Running pre-flight checks.                                                                                                       
[upgrade] Running cluster health checks                                                                                                      
[upgrade] Fetching available versions to upgrade to                                                                                          
[upgrade/versions] Cluster version: v1.23.17                                                                                                 
[upgrade/versions] kubeadm version: v1.24.17                                                                                                 
[upgrade/versions] Target version: 1.24.17                                                                                                   
[upgrade/versions] Latest version in the v1.23 series: 1.24.17
Mon, Mar 18, 10:54 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero added a comment to T347093: [spicerack] Add remote command output to log file.

I was bitten by this recently. I think the proposal made to show at least _something_ in the logs within some sensible limits would makes sense, and definitely help both during cookbook development and later operations.

Mon, Mar 18, 10:05 AM · Infrastructure-Foundations, SRE-tools, cloud-services-team, Spicerack

Fri, Mar 15

aborrero added a project to T327280: [jobs-cli,jobs-api] make API and CLI key/values coherent: User-aborrero.
Fri, Mar 15, 12:19 PM · User-aborrero, Toolforge, cloud-services-team
aborrero changed the status of T320575: [jobs-cli] Allow exporting jobs list in YAML format from Open to In Progress.
Fri, Mar 15, 11:41 AM · Toolforge (Toolforge iteration 07), Patch-For-Review, User-aborrero
aborrero moved T320575: [jobs-cli] Allow exporting jobs list in YAML format from Next to Doing on the User-aborrero board.
Fri, Mar 15, 11:41 AM · Toolforge (Toolforge iteration 07), Patch-For-Review, User-aborrero
aborrero moved T356986: Improve cloudgw filter between VM instances and cloud-private from Doing to Blocked on the User-aborrero board.
Fri, Mar 15, 11:41 AM · Patch-For-Review, User-aborrero, cloud-services-team, Cloud-VPS
aborrero added a comment to T356986: Improve cloudgw filter between VM instances and cloud-private.

ping @cmooney

Fri, Mar 15, 11:40 AM · Patch-For-Review, User-aborrero, cloud-services-team, Cloud-VPS
aborrero closed T356523: [jobs-api] introduce OpenAPI to jobs framework, a subtask of T354745: [jobs-api,buildservice-api,envvars-api] Investigate ways to present our multiple Openapi definitions to a future consolidated CLI client, as Resolved.
Fri, Mar 15, 11:34 AM · Toolforge (Toolforge iteration 07), Patch-For-Review, User-aborrero
aborrero closed T356523: [jobs-api] introduce OpenAPI to jobs framework, a subtask of T358100: [toolforge API] expose all backend APIs OpenAPI specs, as Resolved.
Fri, Mar 15, 11:34 AM · Toolforge (Toolforge iteration 07), Patch-For-Review, User-aborrero
aborrero closed T356523: [jobs-api] introduce OpenAPI to jobs framework as Resolved.
Fri, Mar 15, 11:34 AM · Toolforge (Toolforge iteration 07), User-aborrero
aborrero added a comment to T356523: [jobs-api] introduce OpenAPI to jobs framework.

The openapi JSON can now be fetched from the API:

Fri, Mar 15, 10:55 AM · Toolforge (Toolforge iteration 07), User-aborrero
aborrero closed T360162: Increase Object Storage quota for QRank as Resolved.
aborrero@cloudcontrol1005:~ $ sudo radosgw-admin user info --uid qrank\$qrank
{
    "user_id": "qrank$qrank",
    "display_name": "qrank",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "subusers": [],
    "keys": [],
    "swift_keys": [],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "default_storage_class": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": true,
        "check_on_raw": false,
        "max_size": 8589934592,
        "max_size_kb": 8388608,
        "max_objects": 4096
    },
    "temp_url_keys": [],
    "type": "keystone",
    "mfa_ids": []
}
Fri, Mar 15, 10:38 AM · User-aborrero, cloud-services-team, Cloud-VPS (Quota-requests)
aborrero moved T360162: Increase Object Storage quota for QRank from Backlog to Doing on the User-aborrero board.
Fri, Mar 15, 9:50 AM · User-aborrero, cloud-services-team, Cloud-VPS (Quota-requests)
aborrero changed the status of T360162: Increase Object Storage quota for QRank from Open to In Progress.
Fri, Mar 15, 9:42 AM · User-aborrero, cloud-services-team, Cloud-VPS (Quota-requests)

Thu, Mar 14

aborrero added a comment to T307651: Upgrade Toolforge Kubernetes to version 1.24.

this is scheduled for next monday 2024-03-18.

Thu, Mar 14, 3:58 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero closed T359638: [toolsbeta,infra] upgrade kubernetes to 1.24 as Resolved.

All nodes now running 1.24:

Thu, Mar 14, 1:03 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero closed T359638: [toolsbeta,infra] upgrade kubernetes to 1.24, a subtask of T307651: Upgrade Toolforge Kubernetes to version 1.24, as Resolved.
Thu, Mar 14, 1:03 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero committed rCCKBb73e2be25b71: toolforge.k8s.worker.upgrade: also restart static pods (authored by aborrero).
toolforge.k8s.worker.upgrade: also restart static pods
Thu, Mar 14, 12:23 PM
aborrero committed rCCKB4e139ccd9b2a: kubernetes: fix static pod runtime name (authored by aborrero).
kubernetes: fix static pod runtime name
Thu, Mar 14, 12:02 PM
aborrero committed rCCKB5ced4686ae8e: kubernetes: support .kubelet-keep files in assert_static_pod_path_clean() (authored by aborrero).
kubernetes: support .kubelet-keep files in assert_static_pod_path_clean()
Thu, Mar 14, 11:21 AM
aborrero added a comment to T359638: [toolsbeta,infra] upgrade kubernetes to 1.24.

the first control node upgrade went just fine, doing the next.

Thu, Mar 14, 10:12 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero added a comment to T359638: [toolsbeta,infra] upgrade kubernetes to 1.24.

the upgrade cookbook failed because the problem this patch fixed https://gerrit.wikimedia.org/r/c/cloud/wmcs-cookbooks/+/1010914 and left eh upgrade mid-flight.

Thu, Mar 14, 10:00 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero committed rCCKB3d9954ec2333: wmcs_libs/k8s/kubernetes: fix namespace flags cannot be placed before plugin (authored by aborrero).
wmcs_libs/k8s/kubernetes: fix namespace flags cannot be placed before plugin
Thu, Mar 14, 9:00 AM

Wed, Mar 13

aborrero closed T359927: [cookbook] wmcs.toolforge.k8s.prepare_upgrade: be more flexible checking for deb package components, a subtask of T307651: Upgrade Toolforge Kubernetes to version 1.24, as Resolved.
Wed, Mar 13, 3:05 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero closed T359927: [cookbook] wmcs.toolforge.k8s.prepare_upgrade: be more flexible checking for deb package components as Resolved.
Wed, Mar 13, 3:05 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero committed rCCKB2c6d7b47dd3e: toolforge.k8s.prepare_upgrade: be more flexible when checking repository things (authored by aborrero).
toolforge.k8s.prepare_upgrade: be more flexible when checking repository things
Wed, Mar 13, 3:05 PM
aborrero added a comment to T359649: [jobs-api,infra] upgrade all the existing toolforge jobs to the latest job version.

We did a bit of research today about the size/scope of the problem:

Wed, Mar 13, 10:56 AM · Toolforge

Tue, Mar 12

aborrero created T359927: [cookbook] wmcs.toolforge.k8s.prepare_upgrade: be more flexible checking for deb package components.
Tue, Mar 12, 12:08 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team

Mon, Mar 11

aborrero closed T359798: refresh kube-state-metrics version for k8s 1.24 as Resolved.

This has been deployed in toolsbeta, should be ready for tools.

Mon, Mar 11, 5:02 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero closed T359798: refresh kube-state-metrics version for k8s 1.24, a subtask of T307651: Upgrade Toolforge Kubernetes to version 1.24, as Resolved.
Mon, Mar 11, 5:00 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero added a comment to T338153: toolforge lima-kilo: toolforge-jobs fails to run because it can't create log files.

It's interesting that they are created as root though, we might be missing some restrictions in lima-kilo, in toolforge per se it has the user's user.

Mon, Mar 11, 1:30 PM · Toolforge Jobs framework, cloud-services-team, User-aborrero
aborrero triaged T359816: [cookbook,infra] automate docker image caching workflow as Low priority.
Mon, Mar 11, 1:13 PM · User-aborrero, cloud-services-team, Toolforge
aborrero created T359816: [cookbook,infra] automate docker image caching workflow.
Mon, Mar 11, 1:10 PM · User-aborrero, cloud-services-team, Toolforge
aborrero added a comment to T359650: [jobs-api] Save business models in a DB.

I would be happy to talk about this re-architecture idea. I can share a bit more info about what I tested in the past, and what architecture I had in mind when I first created this, although the code is maybe self-explanatory already.

Mon, Mar 11, 12:06 PM · Toolforge
aborrero changed the status of T359798: refresh kube-state-metrics version for k8s 1.24 from Open to In Progress.
Mon, Mar 11, 10:38 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero changed the status of T359798: refresh kube-state-metrics version for k8s 1.24, a subtask of T307651: Upgrade Toolforge Kubernetes to version 1.24, from Open to In Progress.
Mon, Mar 11, 10:36 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero renamed T359798: refresh kube-state-metrics version for k8s 1.24 from refresh kube-state-metrics to refresh kube-state-metrics version for k8s 1.24.
Mon, Mar 11, 10:36 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero updated the task description for T359798: refresh kube-state-metrics version for k8s 1.24.
Mon, Mar 11, 10:35 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero created T359798: refresh kube-state-metrics version for k8s 1.24.
Mon, Mar 11, 10:30 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero moved T359638: [toolsbeta,infra] upgrade kubernetes to 1.24 from Next to Doing on the User-aborrero board.
Mon, Mar 11, 9:14 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team

Fri, Mar 8

aborrero added a comment to T359650: [jobs-api] Save business models in a DB.

Beware, label values and similar have limitations on what characters they can store.

Fri, Mar 8, 5:03 PM · Toolforge
aborrero created T359642: kubernetes 1.27 requires etcd 3.4.22+ or 3.5.6+.
Fri, Mar 8, 4:00 PM · Toolforge
aborrero added a parent task for T327025: Upgrade Toolforge Kubernetes to version 1.26: T359641: Upgrade Toolforge Kubernetes to version 1.27.
Fri, Mar 8, 3:59 PM · cloud-services-team, Toolforge
aborrero added a subtask for T359641: Upgrade Toolforge Kubernetes to version 1.27: T327025: Upgrade Toolforge Kubernetes to version 1.26.
Fri, Mar 8, 3:59 PM · Toolforge
aborrero created T359641: Upgrade Toolforge Kubernetes to version 1.27.
Fri, Mar 8, 3:58 PM · Toolforge
aborrero added a comment to T339934: [etcd,infra] Find a backup solution for the etcd database.

What if we backup the data to a cinder volume. I know etcd servers run in local storage hypervisors, but I'm not sure if that means they cannot use cinder?

Fri, Mar 8, 3:56 PM · cloud-services-team (FY2023/2024-Q3-Q4), User-aborrero, Toolforge, Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, User-dcaro
aborrero closed T359639: toolforge: verify etcd version is compatible with k8s 1.24 as Resolved.

Per https://v1-24.docs.kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/#prerequisites

Fri, Mar 8, 3:55 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero closed T359639: toolforge: verify etcd version is compatible with k8s 1.24, a subtask of T307651: Upgrade Toolforge Kubernetes to version 1.24, as Resolved.
Fri, Mar 8, 3:54 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero created T359639: toolforge: verify etcd version is compatible with k8s 1.24.
Fri, Mar 8, 3:50 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero changed the status of T307651: Upgrade Toolforge Kubernetes to version 1.24 from Open to In Progress.
Fri, Mar 8, 3:44 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero moved T359638: [toolsbeta,infra] upgrade kubernetes to 1.24 from Next Up to In Progress on the Toolforge (Toolforge iteration 07) board.
Fri, Mar 8, 3:44 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero changed the status of T359638: [toolsbeta,infra] upgrade kubernetes to 1.24 from Open to In Progress.
Fri, Mar 8, 3:44 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero changed the status of T359638: [toolsbeta,infra] upgrade kubernetes to 1.24, a subtask of T307651: Upgrade Toolforge Kubernetes to version 1.24, from Open to In Progress.
Fri, Mar 8, 3:44 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero created T359638: [toolsbeta,infra] upgrade kubernetes to 1.24.
Fri, Mar 8, 3:44 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero changed the status of T307651: Upgrade Toolforge Kubernetes to version 1.24, a subtask of T316107: Upgrade Toolforge Kubernetes to version 1.25, from Open to In Progress.
Fri, Mar 8, 3:43 PM · cloud-services-team, Toolforge
aborrero added a comment to T307651: Upgrade Toolforge Kubernetes to version 1.24.

I plan to start the upgrade on toolsbeta next monday 2024-03-11.

Fri, Mar 8, 3:42 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero closed T359619: toolforge: prepare deb packages for k8s 1.24, a subtask of T307651: Upgrade Toolforge Kubernetes to version 1.24, as Resolved.
Fri, Mar 8, 3:40 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero closed T359619: toolforge: prepare deb packages for k8s 1.24 as Resolved.
Fri, Mar 8, 3:40 PM · Patch-For-Review, Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero changed the status of T359619: toolforge: prepare deb packages for k8s 1.24 from Open to In Progress.
Fri, Mar 8, 1:29 PM · Patch-For-Review, Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero changed the status of T359619: toolforge: prepare deb packages for k8s 1.24, a subtask of T307651: Upgrade Toolforge Kubernetes to version 1.24, from Open to In Progress.
Fri, Mar 8, 1:28 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero moved T359619: toolforge: prepare deb packages for k8s 1.24 from Backlog to Doing on the User-aborrero board.
Fri, Mar 8, 1:28 PM · Patch-For-Review, Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero added a comment to T359620: toolforge: upgrade k8s etcd nodes to debian bookworm.

thanks

Fri, Mar 8, 1:25 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero merged T359620: toolforge: upgrade k8s etcd nodes to debian bookworm into T349207: [infra] Upgrade Toolforge K8s etcd nodes to Bookworm.
Fri, Mar 8, 1:25 PM · cloud-services-team, Kubernetes, Toolforge
aborrero merged task T359620: toolforge: upgrade k8s etcd nodes to debian bookworm into T349207: [infra] Upgrade Toolforge K8s etcd nodes to Bookworm.
Fri, Mar 8, 1:24 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero triaged T359620: toolforge: upgrade k8s etcd nodes to debian bookworm as Low priority.
Fri, Mar 8, 11:56 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero created T359620: toolforge: upgrade k8s etcd nodes to debian bookworm.
Fri, Mar 8, 11:56 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero created T359619: toolforge: prepare deb packages for k8s 1.24.
Fri, Mar 8, 11:47 AM · Patch-For-Review, Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team

Thu, Mar 7

aborrero added a comment to T306790: Set up monitoring for community cronjobs.

I think kubernetes should be logging somewhere all cronjob failures.

Thu, Mar 7, 1:48 PM · User-aborrero, Toolforge, observability
aborrero added a project to T306790: Set up monitoring for community cronjobs: User-aborrero.
Thu, Mar 7, 1:47 PM · User-aborrero, Toolforge, observability
aborrero moved T357977: [toolforge.infra] create fullstack tests from Backlog to Radar on the User-aborrero board.
Thu, Mar 7, 10:25 AM · User-aborrero, Toolforge
aborrero added a project to T357977: [toolforge.infra] create fullstack tests: User-aborrero.
Thu, Mar 7, 10:25 AM · User-aborrero, Toolforge

Wed, Mar 6

aborrero added a comment to T356986: Improve cloudgw filter between VM instances and cloud-private.

the latest iteration on https://gerrit.wikimedia.org/r/c/operations/puppet/+/1007007 brings increased granularity to the firewalling policy.

Wed, Mar 6, 1:33 PM · Patch-For-Review, User-aborrero, cloud-services-team, Cloud-VPS
aborrero moved T356523: [jobs-api] introduce OpenAPI to jobs framework from Next to Doing on the User-aborrero board.
Wed, Mar 6, 1:28 PM · Toolforge (Toolforge iteration 07), User-aborrero

Tue, Mar 5

aborrero moved T319184: Move WMCS servers to 1 single NIC from Doing to Next on the User-aborrero board.
Tue, Mar 5, 3:31 PM · Patch-For-Review, User-aborrero, cloud-services-team, SRE, netops, Infrastructure-Foundations
aborrero moved T357543: clouddb: evaluate moving them into cloud-private from Doing to Radar on the User-aborrero board.
Tue, Mar 5, 3:31 PM · cloud-services-team, netops, User-aborrero, Infrastructure-Foundations
aborrero moved T356986: Improve cloudgw filter between VM instances and cloud-private from Backlog to Doing on the User-aborrero board.
Tue, Mar 5, 3:31 PM · Patch-For-Review, User-aborrero, cloud-services-team, Cloud-VPS
aborrero closed T358476: toolforge k8s: some static pods needs manual restart as Resolved.
Tue, Mar 5, 2:46 PM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team, Kubernetes
aborrero closed T358476: toolforge k8s: some static pods needs manual restart, a subtask of T284656: Toolforge k8s: Migrate workers to Containerd and Bookworm, as Resolved.
Tue, Mar 5, 2:45 PM · User-aborrero, Toolforge (Toolforge iteration 06), Patch-For-Review, cloud-services-team, Kubernetes
aborrero moved T358761: Deploy OVS test setup in codfw1dev from Backlog to Radar on the User-aborrero board.
Tue, Mar 5, 2:45 PM · cloud-services-team (FY2023/2024-Q3-Q4), User-aborrero, Cloud-VPS
aborrero moved T356523: [jobs-api] introduce OpenAPI to jobs framework from Blocked to Next on the User-aborrero board.
Tue, Mar 5, 2:45 PM · Toolforge (Toolforge iteration 07), User-aborrero
aborrero added a comment to T356523: [jobs-api] introduce OpenAPI to jobs framework.

follow up:

  • make sure all jobs (regardless of type) accept the same length for the job name
  • review the /healthz endpoint
  • serve the openapi file via the corresponding API endpoint
Tue, Mar 5, 2:44 PM · Toolforge (Toolforge iteration 07), User-aborrero

Mon, Mar 4

aborrero added a comment to T356986: Improve cloudgw filter between VM instances and cloud-private.

just sent an updated patch with a new approach for the firewall, let me know if that would work or you would rather see it more simplified.

Mon, Mar 4, 4:48 PM · Patch-For-Review, User-aborrero, cloud-services-team, Cloud-VPS

Fri, Mar 1

aborrero committed rCCKBcb1f1c5372cf: toolforge: add restart-static-pods cookbook (authored by aborrero).
toolforge: add restart-static-pods cookbook
Fri, Mar 1, 4:43 PM
aborrero committed rCCKB48f483a619b8: kubernetes: refactor static pod restart logic (authored by aborrero).
kubernetes: refactor static pod restart logic
Fri, Mar 1, 4:42 PM
aborrero moved T358868: Use BGP to announce VM ranges from cloudnet to cloudgw from Backlog to Radar on the User-aborrero board.
Fri, Mar 1, 4:12 PM · cloud-services-team (FY2023/2024-Q3-Q4), User-aborrero, Cloud-VPS
aborrero added a comment to T358868: Use BGP to announce VM ranges from cloudnet to cloudgw.

What we did not try in 2020 was to run bird directly on the box via puppet, without openstack, to learn about upstream routes (to cloudgw).

Fri, Mar 1, 4:11 PM · cloud-services-team (FY2023/2024-Q3-Q4), User-aborrero, Cloud-VPS

Mon, Feb 26

aborrero triaged T357896: ACPI kernel failure on debian installer last step as Low priority.

I haven't checked if the server has the latest firmware updates issued by Dell.

Mon, Feb 26, 3:38 PM · Infrastructure-Foundations, cloud-services-team, User-aborrero, SRE
aborrero triaged T358476: toolforge k8s: some static pods needs manual restart as Medium priority.
Mon, Feb 26, 11:47 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team, Kubernetes
aborrero closed T284656: Toolforge k8s: Migrate workers to Containerd and Bookworm, a subtask of T262562: [infra] Fix the mis-named k8s service in tools and toolsbeta projects, as Resolved.
Mon, Feb 26, 10:55 AM · cloud-services-team, User-Majavah, Toolforge
aborrero closed T284656: Toolforge k8s: Migrate workers to Containerd and Bookworm, a subtask of T307651: Upgrade Toolforge Kubernetes to version 1.24, as Resolved.
Mon, Feb 26, 10:55 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team
aborrero closed T284656: Toolforge k8s: Migrate workers to Containerd and Bookworm, a subtask of T311908: Migrate Toolforge Kubernetes hosts to Debian Bullseye or later, as Resolved.
Mon, Feb 26, 10:55 AM · cloud-services-team, Kubernetes, Toolforge
aborrero closed T284656: Toolforge k8s: Migrate workers to Containerd and Bookworm, a subtask of T355883: Create a pool of NFS-less Toolforge Kubernetes workers, as Resolved.
Mon, Feb 26, 10:55 AM · Patch-For-Review, Toolforge (Toolforge iteration 06)
aborrero closed T284656: Toolforge k8s: Migrate workers to Containerd and Bookworm as Resolved.

This is done:

Mon, Feb 26, 10:54 AM · User-aborrero, Toolforge (Toolforge iteration 06), Patch-For-Review, cloud-services-team, Kubernetes
aborrero updated the task description for T284656: Toolforge k8s: Migrate workers to Containerd and Bookworm.
Mon, Feb 26, 10:52 AM · User-aborrero, Toolforge (Toolforge iteration 06), Patch-For-Review, cloud-services-team, Kubernetes
aborrero created T358476: toolforge k8s: some static pods needs manual restart.
Mon, Feb 26, 10:51 AM · Toolforge (Toolforge iteration 07), User-aborrero, cloud-services-team, Kubernetes