Page MenuHomePhabricator

fnegri (Francesco Negri)
Site Reliability Engineer

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Jul 18 2022, 2:39 PM (90 w, 6 d)
Availability
Available
IRC Nick
dhinus
LDAP User
FNegri
MediaWiki User
FNegri-WMF [ Global Accounts ]

Recent Activity

Fri, Apr 12

fnegri moved T347428: cumin and cloud-vps instances not working from Blocked to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Fri, Apr 12, 2:01 PM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-VPS

Thu, Apr 11

fnegri moved T362233: Decision Request - Toolforge policy agent from Inbox to Discussion on the Cloud Services Proposals board.
Thu, Apr 11, 4:15 PM · Cloud Services Proposals, User-aborrero, cloud-services-team, Toolforge
fnegri added a project to T362233: Decision Request - Toolforge policy agent: Cloud Services Proposals.
Thu, Apr 11, 4:15 PM · Cloud Services Proposals, User-aborrero, cloud-services-team, Toolforge
fnegri moved T362224: Decision request - What to use for toolforge components api task execution from Inbox to Discussion on the Cloud Services Proposals board.
Thu, Apr 11, 3:26 PM · Cloud Services Proposals
fnegri moved T361804: Decision request - Update python team best practices from Inbox to Discussion on the Cloud Services Proposals board.
Thu, Apr 11, 3:25 PM · Cloud Services Proposals
fnegri updated the task description for T346453: [cumin] [openstack] Openstack backend fails when project is not set.
Thu, Apr 11, 10:42 AM · cloud-services-team (FY2023/2024-Q3-Q4), Patch-For-Review, Infrastructure-Foundations, Cloud-VPS, Cumin
fnegri added a comment to T347432: nginx /var/lib/nginx accidentaly mounted on tmpfs in WMCS.

@Andrew do you still want to do a restart of nginx servers? Cumin is now working fine (though only thanks to your manually-applied patch, upstream Cumin is still broken).

Thu, Apr 11, 10:40 AM · Cloud-VPS
fnegri closed T347428: cumin and cloud-vps instances not working as Resolved.

I am marking this task as Resolved, as the commands in the description are now working fine both in cloudcumin1001 and cloud-cumin-03, though they only work because we manually applied the patch https://gerrit.wikimedia.org/r/c/operations/software/cumin/+/868814 to both servers. Getting that patch (or a similar one) properly merged into Cumin is tracked in T346453.

Thu, Apr 11, 10:38 AM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-VPS
fnegri closed T347428: cumin and cloud-vps instances not working, a subtask of T347432: nginx /var/lib/nginx accidentaly mounted on tmpfs in WMCS, as Resolved.
Thu, Apr 11, 10:37 AM · Cloud-VPS
fnegri lowered the priority of T357341: [toolsdb] set gtid_domain_id to 0 from Medium to Low.
Thu, Apr 11, 10:17 AM · cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri updated the task description for T352206: [toolsdb] Upgrade to MariaDB 10.6.
Thu, Apr 11, 9:53 AM · cloud-services-team (FY2023/2024-Q3-Q4), Goal, Data-Services
fnegri added a subtask for T352206: [toolsdb] Upgrade to MariaDB 10.6: T344719: [toolsdb] test failover procedure.
Thu, Apr 11, 9:51 AM · cloud-services-team (FY2023/2024-Q3-Q4), Goal, Data-Services
fnegri added a parent task for T344719: [toolsdb] test failover procedure: T352206: [toolsdb] Upgrade to MariaDB 10.6.
Thu, Apr 11, 9:51 AM · Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri claimed T352206: [toolsdb] Upgrade to MariaDB 10.6.
Thu, Apr 11, 8:38 AM · cloud-services-team (FY2023/2024-Q3-Q4), Goal, Data-Services

Wed, Apr 10

fnegri moved T341060: openstack eqiad1: introduce cloud-private and cloudlb from In progress to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Wed, Apr 10, 8:47 AM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-VPS, Epic, User-aborrero, Goal

Tue, Apr 9

fnegri added a comment to T362051: [component-api] First iteration of the component API.

@dcaro I edited the description of this task to reflect what we discussed in the Toolforge Monthly Meeting.

Tue, Apr 9, 4:19 PM · User-aborrero, Epic, Toolforge
fnegri updated the task description for T362051: [component-api] First iteration of the component API.
Tue, Apr 9, 4:04 PM · User-aborrero, Epic, Toolforge
fnegri moved T346453: [cumin] [openstack] Openstack backend fails when project is not set from Backlog to In progress on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Tue, Apr 9, 2:18 PM · cloud-services-team (FY2023/2024-Q3-Q4), Patch-For-Review, Infrastructure-Foundations, Cloud-VPS, Cumin

Mon, Apr 8

fnegri claimed T346453: [cumin] [openstack] Openstack backend fails when project is not set.
Mon, Apr 8, 9:09 AM · cloud-services-team (FY2023/2024-Q3-Q4), Patch-For-Review, Infrastructure-Foundations, Cloud-VPS, Cumin
fnegri added a project to T334697: Update Help:Access to Toolforge instances with PuTTY and WinSCP: Cloud-VPS.
Mon, Apr 8, 9:06 AM · Cloud-VPS, User-Frostly, good first task, Toolforge, Documentation
fnegri updated the task description for T334697: Update Help:Access to Toolforge instances with PuTTY and WinSCP.
Mon, Apr 8, 9:05 AM · Cloud-VPS, User-Frostly, good first task, Toolforge, Documentation
fnegri updated the task description for T334697: Update Help:Access to Toolforge instances with PuTTY and WinSCP.
Mon, Apr 8, 9:05 AM · Cloud-VPS, User-Frostly, good first task, Toolforge, Documentation

Fri, Apr 5

fnegri added a comment to T359412: [trove] wrong quota_usages values in project tf-infra-test.

The values for in_use and reserved were again showing non-zero values even if there were no active database instances in the tf-infra-test project:

Fri, Apr 5, 10:28 AM · Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4)

Thu, Apr 4

fnegri awarded T361859: Document diffusion->github mirroring to https://github.com/toolforge/ on wikitech a Love token.
Thu, Apr 4, 4:47 PM · User-bd808, Documentation, wikitech.wikimedia.org, Diffusion, Toolforge
fnegri updated the task description for T346453: [cumin] [openstack] Openstack backend fails when project is not set.
Thu, Apr 4, 1:08 PM · cloud-services-team (FY2023/2024-Q3-Q4), Patch-For-Review, Infrastructure-Foundations, Cloud-VPS, Cumin
fnegri added a comment to T358687: "New device" email sent if cookie has expired.

The cookie expiration should be 6 months everywhere.

Thu, Apr 4, 9:27 AM · Community-Tech, MediaWiki-extensions-LoginNotify
fnegri updated the task description for T358687: "New device" email sent if cookie has expired.
Thu, Apr 4, 9:26 AM · Community-Tech, MediaWiki-extensions-LoginNotify
fnegri added a comment to T358687: "New device" email sent if cookie has expired.

This should only happen if your IP address is in a /24 subnet (or /64 for IPv6) that hasn't been used for login in the past 80 days. Can you comment on whether that is likely to be the case?

Thu, Apr 4, 9:20 AM · Community-Tech, MediaWiki-extensions-LoginNotify

Wed, Apr 3

fnegri added a comment to T348887: Decision Request - Incident Response Process.

I have created a draft document that is a WMCS version of this page:
https://docs.google.com/document/d/1lE2Zq_P5wT6nMDxB_ai-_UVw5et6VVSeLornXKD9K3c/edit#heading=h.oiox6yqrxk2h

Wed, Apr 3, 4:20 PM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud Services Proposals
fnegri added a comment to T353891: https://lists.wikimedia.org is often slow to load.

It's very slow for me as well, I hadn't opened it in a while but it was barely usable both yesterday and today.

Wed, Apr 3, 1:13 PM · Upstream, SRE, Performance Issue, Wikimedia-Mailing-lists

Tue, Apr 2

fnegri reassigned T352840: [wmcs-cookbook] increase_quota cookbook fails from fnegri to dcaro.
Tue, Apr 2, 2:44 PM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-VPS
fnegri moved T352840: [wmcs-cookbook] increase_quota cookbook fails from Backlog to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Tue, Apr 2, 1:42 PM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-VPS
fnegri removed a subtask for T356904: [cinder] [toolsdb] Deleting snapshot does not work: T358780: [wmcs-backup] Race condition between backup and cleanup timers.
Tue, Apr 2, 1:20 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri removed a parent task for T358780: [wmcs-backup] Race condition between backup and cleanup timers: T356904: [cinder] [toolsdb] Deleting snapshot does not work.
Tue, Apr 2, 1:20 PM · Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4)
fnegri removed a subtask for T356904: [cinder] [toolsdb] Deleting snapshot does not work: T358774: [wmcs-backup] Backup snapshots of deleted volumes are never cleaned up.
Tue, Apr 2, 1:19 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri removed a parent task for T358774: [wmcs-backup] Backup snapshots of deleted volumes are never cleaned up: T356904: [cinder] [toolsdb] Deleting snapshot does not work.
Tue, Apr 2, 1:19 PM · Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4)
fnegri closed T344717: [toolsdb] test creating a new replica host as Resolved.
Tue, Apr 2, 12:58 PM · Patch-For-Review, Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri removed a subtask for T344717: [toolsdb] test creating a new replica host: T357341: [toolsdb] set gtid_domain_id to 0.
Tue, Apr 2, 12:57 PM · Patch-For-Review, Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri removed a parent task for T357341: [toolsdb] set gtid_domain_id to 0: T344717: [toolsdb] test creating a new replica host.
Tue, Apr 2, 12:57 PM · cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri moved T357341: [toolsdb] set gtid_domain_id to 0 from Backlog to ToolsDB on the Data-Services board.
Tue, Apr 2, 12:56 PM · cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri closed T344717: [toolsdb] test creating a new replica host, a subtask of T335593: ToolsDB: simplify volume chain, as Resolved.
Tue, Apr 2, 12:55 PM · cloud-services-team, Data-Services
fnegri closed T344717: [toolsdb] test creating a new replica host, a subtask of T344420: [toolsdb] Copy s51698__yetkin.wanted_items on the replica from the primary, as Resolved.
Tue, Apr 2, 12:55 PM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, User-dcaro
fnegri closed T344717: [toolsdb] test creating a new replica host, a subtask of T344411: [toolsdb] ToolsDB replication is broken on tools-db-2 (errno 1032) - 2023-08-17, as Resolved.
Tue, Apr 2, 12:55 PM · Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (FY2023/2024-Q1-Q2), User-dcaro
fnegri added a comment to T344717: [toolsdb] test creating a new replica host.

The new replica tools-db-3 is now in sync with the primary. I deleted the old replica tools-db-2.

Tue, Apr 2, 12:55 PM · Patch-For-Review, Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri moved T344420: [toolsdb] Copy s51698__yetkin.wanted_items on the replica from the primary from Blocked to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Tue, Apr 2, 12:49 PM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, User-dcaro
fnegri closed T344420: [toolsdb] Copy s51698__yetkin.wanted_items on the replica from the primary as Declined.

The new replica tools-db-3 is live and includes all tables, so this task is no longer required.

Tue, Apr 2, 12:49 PM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, User-dcaro
fnegri closed T344420: [toolsdb] Copy s51698__yetkin.wanted_items on the replica from the primary, a subtask of T344411: [toolsdb] ToolsDB replication is broken on tools-db-2 (errno 1032) - 2023-08-17, as Declined.
Tue, Apr 2, 12:49 PM · Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (FY2023/2024-Q1-Q2), User-dcaro
fnegri added a comment to T358687: "New device" email sent if cookie has expired.

Adding some more details:

Tue, Apr 2, 8:48 AM · Community-Tech, MediaWiki-extensions-LoginNotify

Fri, Mar 29

fnegri added a comment to T344717: [toolsdb] test creating a new replica host.

After a few attempts, the procedure at https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Toolsdb#Creating_a_new_replica_host should now list all the required steps. I have used it to create tools-db-3 that is currently replicating from tools-db-1.

Fri, Mar 29, 7:24 PM · Patch-For-Review, Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services

Thu, Mar 28

fnegri added a comment to T344412: cloudcumin: support reimage and other operations.

Note that all sre.* cookbooks are no longer installed in cloudcumins (see the discussion in T343894).

Thu, Mar 28, 5:12 PM · Cloud-VPS, Puppet (Puppet 7.0), cloud-services-team, Infrastructure Security
fnegri renamed T344412: cloudcumin: support reimage and other operations from Cloudcumin Gaps to cloudcumin: support reimage and other operations.
Thu, Mar 28, 4:41 PM · Cloud-VPS, Puppet (Puppet 7.0), cloud-services-team, Infrastructure Security
fnegri closed T337848: WMCS-roots wiki replica access as Resolved.

I updated the description of this task noting which parts have been fixed since the description was written.

Thu, Mar 28, 4:23 PM · Data-Services, cloud-services-team, Infrastructure Security
fnegri closed T343330: WMCS cookbooks: provide shared hosts for people without global root privileges as Resolved.

There are still some gaps (T344412: cloudcumin: support reimage and other operations) but I think the main requirement of this task is now satisfied: users without global root can run WMCS cookbooks from cloudcuminXXXX hosts, if they are added to the wmcs-roots group in modules/admin/data/data.yaml.

Thu, Mar 28, 4:12 PM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-VPS
fnegri updated the task description for T337848: WMCS-roots wiki replica access.
Thu, Mar 28, 3:48 PM · Data-Services, cloud-services-team, Infrastructure Security
fnegri updated the task description for T337848: WMCS-roots wiki replica access.
Thu, Mar 28, 3:48 PM · Data-Services, cloud-services-team, Infrastructure Security
fnegri updated the task description for T337848: WMCS-roots wiki replica access.
Thu, Mar 28, 3:28 PM · Data-Services, cloud-services-team, Infrastructure Security
fnegri updated the task description for T337848: WMCS-roots wiki replica access.
Thu, Mar 28, 3:26 PM · Data-Services, cloud-services-team, Infrastructure Security
fnegri moved T337848: WMCS-roots wiki replica access from Needs discussion to Inbox on the cloud-services-team board.

There's no pending discussion at the moment, so I'm moving this task out of "Needs discussion" column and back to the inbox. Feel free to leave a comment if you would like this to be prioritized.

Thu, Mar 28, 3:24 PM · Data-Services, cloud-services-team, Infrastructure Security
fnegri triaged T336905: Supporting AI, LLM, and data models on WMCS as Low priority.
Thu, Mar 28, 3:24 PM · User-aborrero, cloud-services-team
fnegri moved T336905: Supporting AI, LLM, and data models on WMCS from Needs discussion to Inbox on the cloud-services-team board.

There's no pending discussion at the moment, so I'm moving this task out of "Needs discussion" column and back to the inbox. Feel free to leave a comment if you would like this to be prioritized.

Thu, Mar 28, 3:22 PM · User-aborrero, cloud-services-team
fnegri moved T314664: [infra] Decommission the Grid Engine infrastructure from Backlog to In progress on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Thu, Mar 28, 2:47 PM · Toolforge (Toolforge iteration 08), cloud-services-team (FY2023/2024-Q3-Q4), Goal, Patch-For-Review
fnegri moved T348312: [webservice] Error shown when restarting buildpack-based tool from Backlog to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Thu, Mar 28, 2:45 PM · Toolforge (Toolforge iteration 07), cloud-services-team (FY2023/2024-Q3-Q4), Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-User, User-dcaro
fnegri moved T357881: [maintain-kubeusers] Allow setting the requests cpu and mem quota from Backlog to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Thu, Mar 28, 2:45 PM · Toolforge (Toolforge iteration 07), Cloud-Services-Worktype-Project, Cloud-Services-Origin-Team, cloud-services-team (FY2023/2024-Q3-Q4), User-dcaro
fnegri moved T359934: [infra] Archive grid engine related infrastructure tools from Backlog to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Thu, Mar 28, 2:43 PM · Toolforge (Toolforge iteration 07), cloud-services-team (FY2023/2024-Q3-Q4)
fnegri moved T360630: Migrate metricsinfra off buster from Backlog to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Thu, Mar 28, 2:43 PM · Patch-For-Review, cloud-services-team (FY2023/2024-Q3-Q4), Cloud-VPS (Debian Buster Deprecation)

Tue, Mar 26

fnegri changed the status of T344717: [toolsdb] test creating a new replica host from Stalled to In Progress.
Tue, Mar 26, 2:16 PM · Patch-For-Review, Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri closed T356904: [cinder] [toolsdb] Deleting snapshot does not work as Resolved.

Reading again the description, this bug is in effect completely resolved, because the snapshot created during the toolsdb replica procedure can now be deleted.

Tue, Mar 26, 2:15 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri moved T358774: [wmcs-backup] Backup snapshots of deleted volumes are never cleaned up from Backlog to In progress on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Tue, Mar 26, 2:15 PM · Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4)
fnegri changed the status of T344717: [toolsdb] test creating a new replica host, a subtask of T335593: ToolsDB: simplify volume chain, from Stalled to In Progress.
Tue, Mar 26, 2:14 PM · cloud-services-team, Data-Services
fnegri changed the status of T344717: [toolsdb] test creating a new replica host, a subtask of T344411: [toolsdb] ToolsDB replication is broken on tools-db-2 (errno 1032) - 2023-08-17, from Stalled to In Progress.
Tue, Mar 26, 2:14 PM · Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (FY2023/2024-Q1-Q2), User-dcaro
fnegri changed the status of T344717: [toolsdb] test creating a new replica host, a subtask of T344420: [toolsdb] Copy s51698__yetkin.wanted_items on the replica from the primary, from Stalled to In Progress.
Tue, Mar 26, 2:14 PM · cloud-services-team (FY2023/2024-Q3-Q4), Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, User-dcaro
fnegri moved T356904: [cinder] [toolsdb] Deleting snapshot does not work from In progress to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Tue, Mar 26, 2:14 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri closed T356904: [cinder] [toolsdb] Deleting snapshot does not work, a subtask of T344717: [toolsdb] test creating a new replica host, as Resolved.
Tue, Mar 26, 2:14 PM · Patch-For-Review, Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri renamed T356904: [cinder] [toolsdb] Deleting snapshot does not work from [cinder] Deleting snapshot does not work to [cinder] [toolsdb] Deleting snapshot does not work.
Tue, Mar 26, 2:10 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri added a subtask for T344717: [toolsdb] test creating a new replica host: T356904: [cinder] [toolsdb] Deleting snapshot does not work.
Tue, Mar 26, 2:10 PM · Patch-For-Review, Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri added a parent task for T356904: [cinder] [toolsdb] Deleting snapshot does not work: T344717: [toolsdb] test creating a new replica host.
Tue, Mar 26, 2:10 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri lowered the priority of T356904: [cinder] [toolsdb] Deleting snapshot does not work from High to Medium.

This is no longer a blocker for T344717, because the patch https://gerrit.wikimedia.org/r/c/1007636 is now excluding temp volumes from being backed up.

Tue, Mar 26, 2:06 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri renamed T356904: [cinder] [toolsdb] Deleting snapshot does not work from [toolsdb] [cinder] [ceph] Deleting snapshot does not work to [cinder] Deleting snapshot does not work.
Tue, Mar 26, 2:02 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri changed the status of T356904: [cinder] [toolsdb] Deleting snapshot does not work from Stalled to In Progress.
Tue, Mar 26, 2:02 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri removed a subtask for T344717: [toolsdb] test creating a new replica host: T356904: [cinder] [toolsdb] Deleting snapshot does not work.
Tue, Mar 26, 2:02 PM · Patch-For-Review, Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri removed a parent task for T356904: [cinder] [toolsdb] Deleting snapshot does not work: T344717: [toolsdb] test creating a new replica host.
Tue, Mar 26, 2:02 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri changed the status of T356904: [cinder] [toolsdb] Deleting snapshot does not work, a subtask of T344717: [toolsdb] test creating a new replica host, from Stalled to In Progress.
Tue, Mar 26, 2:00 PM · Patch-For-Review, Goal, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri closed T359192: [wmcs-backup] exclude_volumes is matching on IDs instead of names as Resolved.
Tue, Mar 26, 1:45 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4)
fnegri closed T359192: [wmcs-backup] exclude_volumes is matching on IDs instead of names, a subtask of T356904: [cinder] [toolsdb] Deleting snapshot does not work, as Resolved.
Tue, Mar 26, 1:44 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4), Data-Services
fnegri added a subtask for T345337: spicerack: tox fails to install PyYAML using python 3.11 on bookworm: T354670: cleanup apifeatureusage indices on the Cirrus elasticsearch cluster (fix curator).
Tue, Mar 26, 9:36 AM · cloud-services-team (FY2023/2024-Q3-Q4), Patch-For-Review, Infrastructure-Foundations, SRE-tools, Spicerack
fnegri added a parent task for T354670: cleanup apifeatureusage indices on the Cirrus elasticsearch cluster (fix curator): T345337: spicerack: tox fails to install PyYAML using python 3.11 on bookworm.
Tue, Mar 26, 9:36 AM · Data-Platform-SRE (2024.03.25 - 2024.04.14), Patch-For-Review

Mon, Mar 18

fnegri added a comment to T360294: [cloud-vps] creating a new project can override existing DNS entries.

While we find if there's a better way to prevent this, I've added a note to the project creation steps to check for DNS clashes:
https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Projects_lifecycle#Creating_a_new_project

Mon, Mar 18, 2:01 PM · Cloud-VPS
fnegri created T360294: [cloud-vps] creating a new project can override existing DNS entries.
Mon, Mar 18, 10:55 AM · Cloud-VPS

Mar 8 2024

fnegri added a comment to T359564: Advice needed: creating a row for every article across every language Wikipedia in ToolsDB.

Yep, Trove sounds like a better option here. If you think the full database can fit in less than 10GB, you can also try using ToolsDB first, and consider migrating to Trove later. But I would recommend starting directly on Trove.

Mar 8 2024, 2:52 PM · cloud-services-team, Data-Services
fnegri added a comment to T345084: OpenStack API response time gets slower over time .

The alert triggered again yesterday, this time it was caused by a spike in response time for nova-api_backend, that has already ended without any intervention (as far as I know). Attaching a Prometheus graph showing only the affected metrics.

Mar 8 2024, 2:38 PM · cloud-services-team (FY2023/2024-Q3-Q4), User-dcaro, Cloud-VPS

Mar 7 2024

fnegri closed T359412: [trove] wrong quota_usages values in project tf-infra-test as Resolved.

I did reset in_use and reserved values to zero, but I did not truncate the reservations table as it also contains data for all the other projects. I could selectively drop the lines with usage_id matching the tf-infra-project, but I'm not sure if it would be helpful, so I haven't touched the reservations table at all.

Mar 7 2024, 5:31 PM · Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4)
fnegri moved T359412: [trove] wrong quota_usages values in project tf-infra-test from In progress to Done on the cloud-services-team (FY2023/2024-Q3-Q4) board.
Mar 7 2024, 5:31 PM · Cloud-VPS, cloud-services-team (FY2023/2024-Q3-Q4)
fnegri closed T358705: PuppetFailure as Resolved.
Mar 7 2024, 3:30 PM · cloud-services-team
fnegri closed T358702: PuppetFailure Puppet failure on cloudcumin1001:9100 as Resolved.
Mar 7 2024, 3:29 PM · cloud-services-team
fnegri updated the task description for T359531: [trove] move docker images from quay.io to self-hosted registry.
Mar 7 2024, 1:21 PM · Cloud-VPS
fnegri created T359534: [trove] define process for updating docker images.
Mar 7 2024, 1:20 PM · Cloud-VPS
fnegri created T359531: [trove] move docker images from quay.io to self-hosted registry.
Mar 7 2024, 12:58 PM · Cloud-VPS
fnegri added a parent task for T353018: Consider removing Postgres support from Trove: T337396: Better support for Postgres on Trove.
Mar 7 2024, 12:46 PM · PostgreSQL, cloud-services-team, Cloud-VPS
fnegri added a subtask for T337396: Better support for Postgres on Trove: T353018: Consider removing Postgres support from Trove.
Mar 7 2024, 12:46 PM · Cloud-VPS

Mar 6 2024

fnegri closed T359298: Request creation of ipoid-opensearch VPS project as Resolved.

@kostajh project created, I've added you as a member. The default quotas are 8 cores, 16 GB or ram and 80 GB of disk space. Let us know if you need more.

Mar 6 2024, 5:10 PM · Cloud-VPS (Project-requests)