Page MenuHomePhabricator

taavi (Taavi Väänänen)
User

Projects (25)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Feb 24 2019, 3:58 PM (174 w, 3 d)
Availability
Available
IRC Nick
taavi
LDAP User
Majavah
MediaWiki User
Taavi [ Global Accounts ]

Recent Activity

Today

taavi created P30630 (An Untitled Masterwork).
Wed, Jun 29, 8:56 PM
taavi updated subscribers of T311590: ApiEchoUnreadNotificationPages::getUnreadNotificationPagesFromForeign: Unexpected API response from {wiki}.

@tstarling Hey, timing wise this seems to match up with https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/683465/, could yiu take a look?

Wed, Jun 29, 1:31 PM · MediaWiki-extensions-CentralAuth, Growth-Team, Notifications, Wikimedia-production-error
taavi added a subtask for T278392: Storage solution for cross-datacenter tokens: T311590: ApiEchoUnreadNotificationPages::getUnreadNotificationPagesFromForeign: Unexpected API response from {wiki}.
Wed, Jun 29, 1:30 PM · Patch-For-Review, MediaWiki-extensions-OAuth, ConfirmEdit (CAPTCHA extension), MediaWiki-extensions-CentralAuth, Sustainability (MediaWiki-MultiDC), Performance-Team
taavi added a parent task for T311590: ApiEchoUnreadNotificationPages::getUnreadNotificationPagesFromForeign: Unexpected API response from {wiki}: T278392: Storage solution for cross-datacenter tokens.
Wed, Jun 29, 1:30 PM · MediaWiki-extensions-CentralAuth, Growth-Team, Notifications, Wikimedia-production-error
taavi created T311609: deployment-maps-master01 puppet failures.
Wed, Jun 29, 10:46 AM · Product-Infrastructure-Team-Backlog, Beta-Cluster-Infrastructure
taavi created P30599 (An Untitled Masterwork).
Wed, Jun 29, 7:11 AM
taavi added a comment to T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4.

tools-db-1.tools.eqiad1.wikimedia.cloud now is now replicating. The only problem so far is that users and grants didn't seem to get copied over for some reason.

Wed, Jun 29, 6:38 AM · Data-Persistence (Consultation), Cloud-VPS (Debian Stretch Deprecation), cloud-services-team (Kanban), Toolforge, Data-Services

Yesterday

MarioGom awarded T311347: Mark freshly undeleted articles as unreviewed automatically a Love token.
Tue, Jun 28, 8:51 PM · Growth-Team, PageCuration
taavi closed T236420: ToolsDB unstable following unplanned software upgrade as Resolved.

Closing this years-old ticket. The current plan is to upgrade to MariaDB 10.4.

Tue, Jun 28, 5:36 PM · Upstream, cloud-services-team (Kanban), Data-Services
taavi closed T236399: Upgrade mariadb on toolsdb servers to 10.1.44, a subtask of T236420: ToolsDB unstable following unplanned software upgrade, as Declined.
Tue, Jun 28, 5:36 PM · Upstream, cloud-services-team (Kanban), Data-Services
taavi closed T236399: Upgrade mariadb on toolsdb servers to 10.1.44 as Declined.

Closing this years-old ticket. The current plan is to upgrade to MariaDB 10.4.

Tue, Jun 28, 5:36 PM · Data-Services, cloud-services-team (Kanban), Tools
taavi moved T310879: Prepare and check storage layer for pcmwiki from Backlog to Wiki replicas on the Data-Services board.
Tue, Jun 28, 5:26 PM · Data-Services, DBA
taavi moved T310872: Prepare and check storage layer for blkwiki from Backlog to Wiki replicas on the Data-Services board.
Tue, Jun 28, 5:26 PM · Data-Services, DBA
taavi moved T310341: Add some columns of `renameuser_queue` to the replica from Backlog to Wiki replicas on the Data-Services board.
Tue, Jun 28, 5:26 PM · Data-Services, Stewards-and-global-tools, cloud-services-team (Kanban)
taavi moved T309056: Prepare and check storage layer for guwwiktionary from Backlog to Wiki replicas on the Data-Services board.
Tue, Jun 28, 5:26 PM · Data-Services, DBA
taavi moved T310488: View 'altwiki_p.page' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them from Backlog to Wiki replicas on the Data-Services board.
Tue, Jun 28, 5:26 PM · Data-Services
taavi moved T311373: Tool PB: "The MariaDB server is running with the --read-only option so it cannot execute this statement" from Backlog to ToolsDB on the Data-Services board.
Tue, Jun 28, 5:26 PM · cloud-services-team (Kanban), Data-Services, Tools
taavi claimed T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4.
Tue, Jun 28, 4:37 PM · Data-Persistence (Consultation), Cloud-VPS (Debian Stretch Deprecation), cloud-services-team (Kanban), Toolforge, Data-Services
taavi created P30586 (An Untitled Masterwork).
Tue, Jun 28, 4:28 PM
taavi closed T311509: Request increased quota for tools Cloud VPS project as Resolved.
Tue, Jun 28, 3:52 PM · cloud-services-team (Kanban), Data-Services, Toolforge, Cloud-VPS (Quota-requests)
taavi closed T311509: Request increased quota for tools Cloud VPS project, a subtask of T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4, as Resolved.
Tue, Jun 28, 3:52 PM · Data-Persistence (Consultation), Cloud-VPS (Debian Stretch Deprecation), cloud-services-team (Kanban), Toolforge, Data-Services
taavi closed T304708: toolforge: label kubernetes nodes with nfs access as Resolved.
Tue, Jun 28, 3:47 PM · cloud-services-team (Kanban), Toolforge
taavi added a comment to T304708: toolforge: label kubernetes nodes with nfs access.

The patch above works fine for new VMs:

toolsbeta-test-k8s-worker-7    Ready    <none>                 32s    v1.21.8   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=toolsbeta-test-k8s-worker-7,kubernetes.io/os=linux,kubernetes.wmcloud.org/nfs-mounted=true,toolforge.org/nfs-mounted=true

I'm going to write a script to backfill the labels to existing ones.

Tue, Jun 28, 3:41 PM · cloud-services-team (Kanban), Toolforge
taavi added a subtask for T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4: T311509: Request increased quota for tools Cloud VPS project.
Tue, Jun 28, 1:18 PM · Data-Persistence (Consultation), Cloud-VPS (Debian Stretch Deprecation), cloud-services-team (Kanban), Toolforge, Data-Services
taavi added a parent task for T311509: Request increased quota for tools Cloud VPS project: T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4.
Tue, Jun 28, 1:18 PM · cloud-services-team (Kanban), Data-Services, Toolforge, Cloud-VPS (Quota-requests)
taavi created T311509: Request increased quota for tools Cloud VPS project.
Tue, Jun 28, 1:18 PM · cloud-services-team (Kanban), Data-Services, Toolforge, Cloud-VPS (Quota-requests)
taavi committed R2043:ea7f3d33b311: default is now buster (authored by taavi).
default is now buster
Tue, Jun 28, 10:25 AM
taavi claimed T311347: Mark freshly undeleted articles as unreviewed automatically.
Tue, Jun 28, 9:47 AM · Growth-Team, PageCuration

Mon, Jun 27

taavi added a comment to T311184: Request membership in GraphQL and MediaWiki-extensions-GraphQL group for Lectrician1.

Procedural note: Access to the labs/tools/graphql repo should only be granted to maintainers of the graphql Toolforge tool. The process for taking over an abandoned tool is described at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Abandoned_tool_policy.

Mon, Jun 27, 6:33 PM · Gerrit-Privilege-Requests
taavi added a comment to T311011: Request creation of Canasta VPS project.

+1

Mon, Jun 27, 6:31 PM · Cloud-VPS (Project-requests)
taavi added a comment to T311201: Request creation of wikidata-reconciliation VPS project.

+1

Mon, Jun 27, 6:28 PM · Cloud-VPS (Project-requests)
taavi closed T277653: Toolforge: add Debian Buster to the grid and eliminate Debian Stretch as Resolved.
Mon, Jun 27, 6:19 PM · Patch-For-Review, Toolforge, cloud-services-team (Kanban)
taavi closed T277653: Toolforge: add Debian Buster to the grid and eliminate Debian Stretch, a subtask of T275864: Toolforge: migrate to Debian Buster or later, as Resolved.
Mon, Jun 27, 6:18 PM · Cloud-VPS (Debian Stretch Deprecation), Toolforge, cloud-services-team (Kanban), Epic
taavi closed T277653: Toolforge: add Debian Buster to the grid and eliminate Debian Stretch, a subtask of T287900: Switch Toolforge installation of "composer" to use the Debian package, as Resolved.
Mon, Jun 27, 6:18 PM · User-Majavah, Toolforge, Composer
taavi closed T277653: Toolforge: add Debian Buster to the grid and eliminate Debian Stretch, a subtask of T305834: Cloud VPS: drop wmflabs names from profile::resolving::domain_search, as Resolved.
Mon, Jun 27, 6:18 PM · Cloud-VPS, cloud-services-team (Kanban)
taavi renamed T311420: Detect & alert for soon-to-expire CA certs from Detect & email for soon-to-expire CA certs to Detect & alert for soon-to-expire CA certs.
Mon, Jun 27, 5:22 PM · Cloud-VPS, cloud-services-team (Kanban)
taavi closed T311412: Toolforge Puppet CA expired as Resolved.

This might be the reason why this alert triggered:

Mon, Jun 27, 5:22 PM · Toolforge, cloud-services-team (Kanban)
taavi added a comment to T311373: Tool PB: "The MariaDB server is running with the --read-only option so it cannot execute this statement".

Hi! As far as I can see, tools-db is not currently in read only mode. Your code file is quite large, do you have the exact statement that is failing?

Mon, Jun 27, 4:17 PM · cloud-services-team (Kanban), Data-Services, Tools
taavi edited P30459 (An Untitled Masterwork).
Mon, Jun 27, 3:20 PM
taavi changed the edit policy for P30459 (An Untitled Masterwork).
Mon, Jun 27, 3:17 PM
taavi created P30459 (An Untitled Masterwork).
Mon, Jun 27, 3:17 PM
taavi claimed T311412: Toolforge Puppet CA expired.
Mon, Jun 27, 3:10 PM · Toolforge, cloud-services-team (Kanban)
taavi triaged T311412: Toolforge Puppet CA expired as Unbreak Now! priority.
Mon, Jun 27, 1:37 PM · Toolforge, cloud-services-team (Kanban)

Sat, Jun 25

taavi added a comment to T311331: +2 for Dreamy_Jazz in mediawiki/extensions/CheckUser.

Support. I don't consider the lack of existing reviews to be a problem, @Dreamy_Jazz's previous contributions and on-wiki functionary roles clearly demonstrate that we can trust them to make good decisions about patches in areas they're familiar with and ask for help if they're unsure.

Sat, Jun 25, 7:16 AM · Gerrit-Privilege-Requests, CheckUser

Fri, Jun 24

taavi awarded T311314: Shell access request for @demon a Barnstar token.
Fri, Jun 24, 7:44 PM · SRE, SRE-Access-Requests
taavi closed T311302: Request increased quota for devtools Cloud VPS project as Resolved.

Done!

Fri, Jun 24, 12:44 PM · cloud-services-team (Kanban), Release-Engineering-Team, Cloud-VPS (Quota-requests)
taavi closed T311302: Request increased quota for devtools Cloud VPS project, a subtask of T311240: Puppet agent failing on deploy-1004.devtools.eqiad1.wikimedia.cloud, as Resolved.
Fri, Jun 24, 12:43 PM · cloud-services-team (Kanban), VPS-Projects
taavi claimed T311302: Request increased quota for devtools Cloud VPS project.
Fri, Jun 24, 12:41 PM · cloud-services-team (Kanban), Release-Engineering-Team, Cloud-VPS (Quota-requests)
taavi closed T221268: Remove old letsencrypt puppet module as Resolved.

https://gerrit.wikimedia.org/r/c/operations/puppet/+/655762

Fri, Jun 24, 8:17 AM · Traffic-Icebox, Infrastructure-Foundations, Puppet, SRE

Thu, Jun 23

taavi updated the task description for T306099: Cloud VPS "tools" project Stretch deprecation.
Thu, Jun 23, 6:05 PM · cloud-services-team (Kanban), Toolforge, Cloud-VPS (Debian Stretch Deprecation)
taavi updated the task description for T306099: Cloud VPS "tools" project Stretch deprecation.
Thu, Jun 23, 5:55 PM · cloud-services-team (Kanban), Toolforge, Cloud-VPS (Debian Stretch Deprecation)
taavi updated the task description for T306099: Cloud VPS "tools" project Stretch deprecation.
Thu, Jun 23, 5:30 PM · cloud-services-team (Kanban), Toolforge, Cloud-VPS (Debian Stretch Deprecation)
taavi added a comment to T310974: Extend PageTriageMaxAge for unpatrolled articles at enwiki.

@JJMC89 could we have an answer please?

Thu, Jun 23, 8:03 AM · Community-consensus-needed, Patch-For-Review, MW-1.39-notes (1.39.0-wmf.18; 2022-06-27), Wikimedia-Site-requests, English-Wikipedia-New-Pages-Patrol

Wed, Jun 22

taavi added a comment to T293627: l10n-bot-watcher permissions missing from mediawiki/extensions/VisualEditor.

Is this still needed?

Wed, Jun 22, 5:39 PM · VisualEditor, Gerrit-Privilege-Requests
taavi moved T293627: l10n-bot-watcher permissions missing from mediawiki/extensions/VisualEditor from Backlog to Needs discussion on the Gerrit-Privilege-Requests board.
Wed, Jun 22, 5:38 PM · VisualEditor, Gerrit-Privilege-Requests
taavi moved T308278: Request membership in "mediawiki/extensions/OpenBadges" group for tfellows from Backlog to Resolved on the Gerrit-Privilege-Requests board.
Wed, Jun 22, 5:37 PM · Gerrit-Privilege-Requests
taavi closed T308278: Request membership in "mediawiki/extensions/OpenBadges" group for tfellows as Resolved.

Done, since this is not a WMF deployed extension and this task had an endorsement for someone who already has rights to that extension. Sorry for the delay!

Wed, Jun 22, 5:37 PM · Gerrit-Privilege-Requests
taavi moved T303147: Request membership in MediaWiki-extensions-JsonData group for robla from Backlog to Resolved on the Gerrit-Privilege-Requests board.
Wed, Jun 22, 5:35 PM · Gerrit-Privilege-Requests, MediaWiki-extensions-JsonData
taavi closed T303147: Request membership in MediaWiki-extensions-JsonData group for robla as Resolved.

Done. Sorry for the delay!

Wed, Jun 22, 5:35 PM · Gerrit-Privilege-Requests, MediaWiki-extensions-JsonData
taavi claimed T303147: Request membership in MediaWiki-extensions-JsonData group for robla.
Wed, Jun 22, 5:33 PM · Gerrit-Privilege-Requests, MediaWiki-extensions-JsonData
taavi closed T281275: upgrade cloud-vps openstack to Openstack version 'Wallaby' as Resolved.
Wed, Jun 22, 5:29 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (Kanban)
taavi closed T281275: upgrade cloud-vps openstack to Openstack version 'Wallaby', a subtask of T296561: upgrade cloud-vps openstack to Openstack version 'Xena', as Resolved.
Wed, Jun 22, 5:29 PM · cloud-services-team (Kanban), Cloud-VPS
taavi closed T304694: upgrade codfw1dev to wallaby, a subtask of T281275: upgrade cloud-vps openstack to Openstack version 'Wallaby', as Resolved.
Wed, Jun 22, 5:28 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (Kanban)
taavi closed T304694: upgrade codfw1dev to wallaby as Resolved.
Wed, Jun 22, 5:28 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS
taavi closed T302178: prometheus-openstack-exporter No module named 'urlparse' as Resolved.
Wed, Jun 22, 5:28 PM · Cloud-Services-Origin-Team, Cloud-Services-Worktype-Project, User-dcaro, Patch-For-Review, Cloud-VPS, cloud-services-team (Kanban)
taavi closed T302178: prometheus-openstack-exporter No module named 'urlparse', a subtask of T281276: Upgrade cloud-vps openstack hosts to Debian 'Bullseye', as Resolved.
Wed, Jun 22, 5:28 PM · Cloud-VPS, cloud-services-team (Kanban)
taavi closed T302178: prometheus-openstack-exporter No module named 'urlparse', a subtask of T302050: prometheus-openstack-exporter in Bullseye, as Resolved.
Wed, Jun 22, 5:28 PM · cloud-services-team (Kanban)
taavi closed T302381: cloudcontrol1005 - create alert for prometheus-openstack-exporter service, a subtask of T302369: cloudcontrol1003 - Check for VMs leaked by the nova-fullstack test, as Resolved.
Wed, Jun 22, 5:24 PM · Patch-For-Review, Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, cloud-services-team (Kanban), User-dcaro
taavi closed T302381: cloudcontrol1005 - create alert for prometheus-openstack-exporter service as Resolved.

This is covered by the generic Prometheus target down alert.

Wed, Jun 22, 5:24 PM · Cloud-Services-Worktype-Maintenance, Cloud-Services-Origin-Team, cloud-services-team (Kanban), User-dcaro
taavi added a comment to T277653: Toolforge: add Debian Buster to the grid and eliminate Debian Stretch.

The following SGE jobs are still running on the exec grid:

tools.alchimista 8787914 tlgm
tools.ammarbot 9058295 cine_review
tools.avicbot 400742 uaaby5min
tools.avicbot 6444029 avicbotirc
tools.cewbot 3280296 cron-tools.cewbot-IRC
tools.earwigbot 9782320 earwigbot
tools.germancon-mobile 400540 germancon-mobile-parser
tools.hat-collector 3965351 go
tools.listeria 3216144 bot
tools.nokib-bot 1593642 daily
tools.patest 6882116 script_wui
tools.phetools 2742445 ws_ocr_daemon
tools.phetools 2742447 verify_match
tools.phetools 2742450 extract_text_layer
tools.sdzerobot 6303775 stream
tools.signature-checker 2297341 cron-20170515.signature_check.wikinews
tools.signature-checker 2297344 cron-20170515.signature_check.wikisource
tools.signature-checker 2297345 cron-20170515.signature_check.wikiversity
tools.signature-checker 4615899 cron-20170515.signature_check.zh
tools.signature-checker 5086089 cron-20170515.signature_check.simple
tools.signature-checker 5969401 cron-20170515.signature_check.wiktionary
tools.signature-checker 6091396 cron-20170515.signature_check.zh-classical
tools.signature-manquante-bot 4769872 signature-manquante
tools.telegram-wikilinksbot 5699376 wikilinksbot
tools.toc 1826241 cron-20170915.topic_list.ja
tools.toc 3609056 cron-20170915.topic_list.wikinews
tools.toc 3609088 cron-20170915.topic_list.wikisource
tools.toc 3609172 cron-20170915.topic_list.wikiversity
tools.toc 5086111 cron-20170915.topic_list.wiktionary
tools.toc 5287268 cron-20170915.topic_list.moegirl
tools.urbanecmbot 2811129 patrolAfterPatrol
tools.urbanecmbot 6443111 patrolSandbox
tools.wikidata-todo 2840123 wc_all
tools.wikidata-todo 2840124 wc_rc
tools.wikitanvirbot 1366931 doubredi
tools.wikitanvirbot 5512585 sand-wbbn
tools.wikitanvirbot 5512746 corona
tools.wikitanvirbot 5816767 sand-wtbn

I'm going to kill those sometime tomorrow (so 2022-06-23).

Wed, Jun 22, 5:14 PM · Patch-For-Review, Toolforge, cloud-services-team (Kanban)
taavi created T311149: neutron agents losing RabbitMQ connectivity don't crash properly.
Wed, Jun 22, 2:48 PM · Cloud-VPS, cloud-services-team (Kanban)

Tue, Jun 21

taavi reopened T309375: Requesting access to contint-admins for taavi as "Open".

Hi @SLyngshede-WMF, please also add myself to the ciadmin ldap group as requested in the task description. Thanks!

Tue, Jun 21, 6:59 AM · SRE, SRE-Access-Requests
Aklapper renamed taavi from Majavah to taavi.
Tue, Jun 21, 6:40 AM

Mon, Jun 20

taavi changed the status of T302870: Grant cn=nda some sort of read only access to Netbox from Stalled to Open.
Mon, Jun 20, 4:07 PM · SRE, Infrastructure-Foundations, netbox
taavi created T310999: please rename my phabricator account `Majavah` -> `taavi`.
Mon, Jun 20, 1:47 PM · Phabricator
taavi updated taavi.
Mon, Jun 20, 1:44 PM

Sun, Jun 19

taavi claimed T286301: Scale up metricsinfra prometheus beyond one Prometheus instance (Thanos/Cortex/similar).
Sun, Jun 19, 4:38 PM · Patch-For-Review, Cloud-VPS
taavi committed rCMPC84e496b2411d: add external_labels support (authored by taavi).
add external_labels support
Sun, Jun 19, 4:34 PM

Sat, Jun 18

Legoktm awarded T309375: Requesting access to contint-admins for taavi a Like token.
Sat, Jun 18, 1:10 AM · SRE, SRE-Access-Requests

Fri, Jun 17

taavi closed T285055: Allow project members/admins to ack/silence alerts of that project, a subtask of T266050: Build Prometheus service for use by all Cloud VPS projects and their instances, as Resolved.
Fri, Jun 17, 7:18 PM · cloud-services-team (Kanban), Cloud-VPS
taavi closed T285055: Allow project members/admins to ack/silence alerts of that project as Resolved.
Fri, Jun 17, 7:18 PM · User-Majavah, Cloud-VPS
taavi added a comment to T288108: Figure out how to deal with security groups when rolling out metricsinfra scraping.

It does a bit, but that can be worked around if needed. Previously I'd been working with the assumption that we do want to keep basic health metrics for all projects. If that's not something we want, that's fine, I just want that explicitely documented somewhere.

Fri, Jun 17, 2:30 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS
taavi added a comment to T288108: Figure out how to deal with security groups when rolling out metricsinfra scraping.

Decided to be bold and created the second address:

taavi@cloudcontrol1004 ~ $ os port create --network 7425e328-560c-4f00-8e99-706f3fb90bb4 --description "reserved address for monitoring" "metricsinfra reserved address #2"

This means we now have the following:

taavi@cloudcontrol1004 ~ $ os port list | grep metricsinfra
| 02099d16-afab-4f27-83fc-8ffe0663e421 | metricsinfra reserved address #2                   | fa:16:3e:f8:4f:36 | ip_address='172.16.6.65', subnet_id='a69bdfad-d7d2-4cfa-8231-3d6d3e0074c9'   | DOWN   |
| 2e67a486-e840-4800-b974-d9220f5e107a | metricsinfra reserved address #1                   | fa:16:3e:c1:4d:69 | ip_address='172.16.0.229', subnet_id='a69bdfad-d7d2-4cfa-8231-3d6d3e0074c9'  | ACTIVE |
Fri, Jun 17, 1:17 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS
taavi placed T309375: Requesting access to contint-admins for taavi up for grabs.
Fri, Jun 17, 12:53 PM · SRE, SRE-Access-Requests

Thu, Jun 16

taavi added a subtask for T310799: Upgrade metricsinfra prometheus to bullseye: T288108: Figure out how to deal with security groups when rolling out metricsinfra scraping.
Thu, Jun 16, 7:15 PM · cloud-services-team (Kanban), Cloud-VPS
taavi added a parent task for T288108: Figure out how to deal with security groups when rolling out metricsinfra scraping: T310799: Upgrade metricsinfra prometheus to bullseye.
Thu, Jun 16, 7:15 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS
taavi committed rODIT4702ffd90341: perl: add libfile-slurp-perl package (authored by taavi).
perl: add libfile-slurp-perl package
Thu, Jun 16, 6:52 PM
taavi moved T310821: toolforge: Provide a node 16 image from Inbox to Needs discussion on the cloud-services-team (Kanban) board.
Thu, Jun 16, 5:02 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge (Software install/update)
taavi added a project to T310821: toolforge: Provide a node 16 image: cloud-services-team (Kanban).
Thu, Jun 16, 5:01 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge (Software install/update)
taavi created T310821: toolforge: Provide a node 16 image.
Thu, Jun 16, 5:01 PM · Patch-For-Review, cloud-services-team (Kanban), Toolforge (Software install/update)
taavi updated the task description for T266050: Build Prometheus service for use by all Cloud VPS projects and their instances.
Thu, Jun 16, 4:07 PM · cloud-services-team (Kanban), Cloud-VPS
taavi reassigned T288108: Figure out how to deal with security groups when rolling out metricsinfra scraping from taavi to Andrew.

@Andrew Hello! I think T310799: Upgrade metricsinfra prometheus to bullseye gives me the reason I needed to finally get this done. In short:

  • I want to create a new pair of instances, metricsinfra-prometheus-2 and metricsinfra-prometheus-3, with the g3.cores4.ram8.disk20 flavor. Those new instances should have access to the node-exporter port on all VMs on all projects.
  • In addition to Neutron security groups, the new instances also need to be added to the profile::wmcs::instance::metricsinfra_prometheus_nodes hiera key.
  • The existing instance was given a reserved address in T250206#6056467. We can re-use that for one of the instances and create a new similarly reserved port for the other one. That gives us stable IPs that don't change the next time we're upgrading these VMs.
  • I believe we can update wmfkeystonehooks to create the needed security group rules by default and write some scripts to backfill it to existing projects.
Thu, Jun 16, 4:05 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS
taavi created T310802: Request increased quota for metricsinfra Cloud VPS project.
Thu, Jun 16, 2:25 PM · cloud-services-team (Kanban), Cloud-VPS (Quota-requests)
taavi created T310799: Upgrade metricsinfra prometheus to bullseye.
Thu, Jun 16, 2:12 PM · cloud-services-team (Kanban), Cloud-VPS

Wed, Jun 15

taavi created P29850 (An Untitled Masterwork).
Wed, Jun 15, 5:22 PM
taavi closed T305308: Buster grid problem: some perl modules are missing as Resolved.

Hi and sorry for the delay!

Wed, Jun 15, 4:44 PM · Toolforge (Software install/update), cloud-services-team (Kanban)
taavi added a comment to T284767: Toolforge: migrate cron servers to Debian Buster.

I wanted to use this to ensure we only have one active cron runner:

# if this is not the active cron runner, block tool crons for easier migration between nodes,
# but allow root owned crons (most imporantly puppet runs) to still run as intended
# for more details, see crontab(1)
file { '/etc/cron.allow':
    ensure  => $is_active.bool2str('absent', 'file'),
    content => "root\n",
}

That does not work. /etc/cron.allow controls who can use the crontab command, but the cron daemon itself ignores it.

Wed, Jun 15, 4:24 PM · Patch-For-Review, Toolforge, cloud-services-team (Kanban)
taavi updated the task description for T267082: Rebuild Toolforge servers that should not have NFS mounted (and with affinity).
Wed, Jun 15, 3:32 PM · Toolforge, cloud-services-team (Kanban)
taavi claimed T284767: Toolforge: migrate cron servers to Debian Buster.
Wed, Jun 15, 2:25 PM · Patch-For-Review, Toolforge, cloud-services-team (Kanban)
taavi closed T278541: Toolforge: migrate redis servers to Debian Buster or later, a subtask of T275864: Toolforge: migrate to Debian Buster or later, as Resolved.
Wed, Jun 15, 2:23 PM · Cloud-VPS (Debian Stretch Deprecation), Toolforge, cloud-services-team (Kanban), Epic