Page MenuHomePhabricator

aborrero (arturo)
SRE at Wikimedia Cloud Services Team

Projects (6)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Sunday

  • Clear sailing ahead.

User Details

User Since
Oct 23 2017, 12:19 PM (135 w, 4 d)
Availability
Available
IRC Nick
arturo
LDAP User
Arturo Borrero Gonzalez
MediaWiki User
ABorrero (WMF) [ Global Accounts ]

I'm Arturo Borrero Gonzalez from Spain (Seville). I'm Site Reliability Engineer (SRE) in the Wikimedia Cloud Services Team, a Wikimedia Foundation staff.

You may find me in some FLOSS projects, like Netfilter and Debian.

Recent Activity

Yesterday

aborrero closed T253816: *.toolforge.org hostnames unexpectly treated as tools.wmflabs.org when the URL's path starts with a match to a grid engine webservice as Resolved.

Thanks @MusikAnimal and @bd808 for this!

Thu, May 28, 12:32 PM · cloud-services-team (Kanban), Toolforge
aborrero added a comment to T253702: Figure out storing images for PAWS.

@aborrero I presume you'll want access as well?

Thu, May 28, 10:54 AM · cloud-services-team (Kanban), PAWS

Wed, May 27

aborrero added a comment to T253738: ToolsDB: master crashed, replica having consistency issues.

BTW, for the record, some grafana metrics for context:

Wed, May 27, 10:06 AM · Data-Services, cloud-services-team (Kanban)
aborrero updated the task description for T253738: ToolsDB: master crashed, replica having consistency issues.
Wed, May 27, 9:52 AM · Data-Services, cloud-services-team (Kanban)
aborrero created T253738: ToolsDB: master crashed, replica having consistency issues.
Wed, May 27, 9:51 AM · Data-Services, cloud-services-team (Kanban)

Tue, May 26

aborrero added a comment to T246122: Upgrade the Toolforge Kubernetes cluster to v1.16.
NOTE: kubeadm suggest we should upgrade etcd, but 3.2 is what we have for now, in both Debian and the WMF repos.
Tue, May 26, 10:04 AM · Patch-For-Review, Epic, Toolforge, cloud-services-team (Kanban), Kubernetes
aborrero updated the task description for T246122: Upgrade the Toolforge Kubernetes cluster to v1.16.
Tue, May 26, 9:23 AM · Patch-For-Review, Epic, Toolforge, cloud-services-team (Kanban), Kubernetes
aborrero updated the task description for T246122: Upgrade the Toolforge Kubernetes cluster to v1.16.
Tue, May 26, 9:23 AM · Patch-For-Review, Epic, Toolforge, cloud-services-team (Kanban), Kubernetes
aborrero closed T250867: Script the process of upgrading a node with kubeadm to 1.16.9, a subtask of T246122: Upgrade the Toolforge Kubernetes cluster to v1.16, as Resolved.
Tue, May 26, 9:22 AM · Patch-For-Review, Epic, Toolforge, cloud-services-team (Kanban), Kubernetes
aborrero closed T250867: Script the process of upgrading a node with kubeadm to 1.16.9 as Resolved.

Closing task for now, this seems done.

Tue, May 26, 9:22 AM · Patch-For-Review, Toolforge, cloud-services-team (Kanban), Kubernetes

Mon, May 25

aborrero closed T247972: Cloud DNS: fix inconsistent ownership of reverse domains for openstack floating ip networks as Resolved.

Ok, after a bit of back and forth, this works now:

Mon, May 25, 5:19 PM · cloud-services-team (Kanban)
aborrero closed T247972: Cloud DNS: fix inconsistent ownership of reverse domains for openstack floating ip networks, a subtask of T247971: Cloud DNS: update markmonitor entries, as Resolved.
Mon, May 25, 5:19 PM · cloud-services-team (Kanban)
aborrero closed T247972: Cloud DNS: fix inconsistent ownership of reverse domains for openstack floating ip networks, a subtask of T245173: CloudVPS: DNS improvements, as Resolved.
Mon, May 25, 5:19 PM · cloud-services-team (Kanban), Epic
aborrero added a comment to T247972: Cloud DNS: fix inconsistent ownership of reverse domains for openstack floating ip networks.

Apparently the string gets evaluated against this regex: RE_ZONENAME = r'^(?!.{255,})(?:(?!\-)[A-Za-z0-9_\-]{1,63}(?<!\-)\.)+\Z' which doesn't allow the 0/29.

Mon, May 25, 12:55 PM · cloud-services-team (Kanban)
aborrero added a comment to T247972: Cloud DNS: fix inconsistent ownership of reverse domains for openstack floating ip networks.
root@cloudcontrol2001-dev:~# openstack zone create --email root@wmflabs.org --type PRIMARY --ttl 3600 --description "floating IPs subnet" 0/29.57.15.185.in-addr.arpa.
Provided object is not valid. Got a ValueError error with message Domain 0/29.57.15.185.in-addr.arpa. is not match
Mon, May 25, 12:47 PM · cloud-services-team (Kanban)

Thu, May 21

aborrero added a comment to T253241: Add helm3 to the component repo.

So, the package is in the repo already, in this version 3.2.0-1+deb10u1, in the main version.. Let me send a patch now.

Thu, May 21, 2:24 PM · Toolforge, cloud-services-team (Kanban), PAWS
aborrero added a comment to T253241: Add helm3 to the component repo.

Well, I just found https://gerrit.wikimedia.org/r/admin/projects/operations/debs/helm3

Thu, May 21, 2:21 PM · Toolforge, cloud-services-team (Kanban), PAWS
aborrero added a comment to T253241: Add helm3 to the component repo.

I cannot find any .deb package with helm. Do you think we should package it ourselves or just let puppet download it?

Thu, May 21, 2:19 PM · Toolforge, cloud-services-team (Kanban), PAWS
aborrero added a comment to T211096: PAWS: Rebuild and upgrade Kubernetes.

it turns out the ownership dancing is not obvious from the generic transfer docs (https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/DNS/Designate#DNS_zone_creating_/_transfers which works well for reverse zones etc), so I created a specific section for this use case: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/DNS/Designate#svc_zone_in_.wikimedia.cloud_domain I think we will have this specific use case more often than the other.

Thu, May 21, 12:51 PM · Patch-For-Review, Toolforge, Epic, Goal, cloud-services-team (Kanban), PAWS
aborrero closed T247422: Update Tesseract on Toolforge to v4.1.0 as Resolved.

@bd808 the tesseract-ocr-all is a metapackage, it has no content but just dependencies to all the lang-specific packages.
The lang-specific packages don't have a strong dependency on the version of tesseract. It seems we can have tesseract-ocr 4.1.1 and the lang-specific pack in 4.0 or whatever. Actually, there are no lang-specific packages in 4.1.1 in Debian.

Thu, May 21, 10:50 AM · cloud-services-team (Kanban), Toolforge (Software install/update)
aborrero closed T247422: Update Tesseract on Toolforge to v4.1.0, a subtask of T244100: Spike: New/Improved OCR tool [8 hours], as Resolved.
Thu, May 21, 10:50 AM · Community-Tech (Kanban-2019-20-Q4), Internet-Archive

Wed, May 20

aborrero added a comment to T247972: Cloud DNS: fix inconsistent ownership of reverse domains for openstack floating ip networks.

TODO: verify that we can actually resolve stuff like dig 1.57.15.185.in-addr.arpa. Will probably need to take a look at designate @ codfw1dev

Wed, May 20, 3:59 PM · cloud-services-team (Kanban)
aborrero closed T252762: tools/toolsbeta: improve acme-chief integration as Declined.

I'm unblocked now. Closing this task in favor of whatever we decide on T252721: cloud-vps solution for Let's Encrypt.

Wed, May 20, 3:35 PM · Acme-chief, cloud-services-team (Kanban)
aborrero added a comment to T247422: Update Tesseract on Toolforge to v4.1.0.

hey @kaldari I finally deployed tesseract v4.1.0 in Toolforge.

Wed, May 20, 10:01 AM · cloud-services-team (Kanban), Toolforge (Software install/update)
aborrero added a comment to T247972: Cloud DNS: fix inconsistent ownership of reverse domains for openstack floating ip networks.

ok, will take a look soon.

Wed, May 20, 9:01 AM · cloud-services-team (Kanban)

Tue, May 19

aborrero closed T251295: Plan the integration of new WMCS naming schemes into PAWS, a subtask of T211096: PAWS: Rebuild and upgrade Kubernetes, as Resolved.
Tue, May 19, 11:30 AM · Patch-For-Review, Toolforge, Epic, Goal, cloud-services-team (Kanban), PAWS
aborrero closed T251295: Plan the integration of new WMCS naming schemes into PAWS as Resolved.

Ok, my understanding is we will be using paws.wmcloud.org for the new service, right?

Tue, May 19, 11:30 AM · Toolforge, cloud-services-team (Kanban), PAWS
aborrero added a comment to T244222: CloudVPS: hiera refactor.

Shall we move on and refactor the labs/private.git hiera tree too?

Tue, May 19, 11:28 AM · Patch-For-Review, Epic, cloud-services-team (Kanban)

Mon, May 18

aborrero added a comment to T252762: tools/toolsbeta: improve acme-chief integration.

Sorry, I forgot what thing we agreed on? Anyway, I agree with everything that simplifies the setup :-)

Mon, May 18, 1:01 PM · Acme-chief, cloud-services-team (Kanban)
aborrero added a comment to T252397: wdqs-wmil-tutorial: make sure it uses the new domain toolforge.org.

Dear @aborrero ,
Here is Sasha Tamarin (username:spontans),
I've requested an account for the maintenance of https://wdqs-wmil-tutorial.toolforge.org/.
So I want to confirm that I'm contributing to this website.
Thanks,
Sasha

Mon, May 18, 10:24 AM · Tools, Toolforge

Thu, May 14

aborrero added a comment to T120225: Toolforge: correctly envelope forwarded email.

I just opened T252762: tools/toolsbeta: improve acme-chief integration which is blocking a correct deployment of the mail server in toolsbeta for testing.

Thu, May 14, 12:03 PM · Patch-For-Review, Mail, Toolforge
aborrero created T252762: tools/toolsbeta: improve acme-chief integration.
Thu, May 14, 11:59 AM · Acme-chief, cloud-services-team (Kanban)

Wed, May 13

aborrero added a comment to T250867: Script the process of upgrading a node with kubeadm to 1.16.9.

I created some docs for this script in https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Kubernetes/Upgrading_Kubernetes

Wed, May 13, 11:29 AM · Patch-For-Review, Toolforge, cloud-services-team (Kanban), Kubernetes
aborrero added a comment to T250863: Upgrade calico to a more recent version (current is 3.14.0).

@aborrero I *think* this will mean we won't need to worry about the iptables fixup anymore once this is the standard in Toolforge, right?

Wed, May 13, 8:46 AM · Toolforge, cloud-services-team (Kanban), Kubernetes

Tue, May 12

aborrero added a comment to T252039: Story idea for Blog: Share about the 2020 Kuberentes cluster migration project.

I think the 3 of us should be listed as authors! :-) or coauthor or whatever term better reflect that we collaborated to produce the article.

Tue, May 12, 9:20 AM · Technical-blog-posts

Mon, May 11

aborrero added a comment to T251753: Request creation of wikidata-query-service-tutorial VPS project.

Awesome! Great job! :-) I'm really glad of this.

Mon, May 11, 11:13 AM · Cloud-VPS (Project-requests)
aborrero created T252397: wdqs-wmil-tutorial: make sure it uses the new domain toolforge.org.
Mon, May 11, 11:12 AM · Tools, Toolforge
aborrero added a comment to T252039: Story idea for Blog: Share about the 2020 Kuberentes cluster migration project.

Thanks for the review!

Mon, May 11, 11:03 AM · Technical-blog-posts

Fri, May 8

aborrero added a comment to T120225: Toolforge: correctly envelope forwarded email.

I tried setting up the testing server. There is something going on with letsencrypt:

Fri, May 8, 12:44 PM · Patch-For-Review, Mail, Toolforge
aborrero added a comment to T252039: Story idea for Blog: Share about the 2020 Kuberentes cluster migration project.

Sent initial draft to @srodlund and @Bstorm

Fri, May 8, 11:15 AM · Technical-blog-posts

Thu, May 7

Bstorm awarded T251297: Refactor the toolforge::k8s::kubeadm* modules a Party Time token.
Thu, May 7, 5:45 PM · Toolforge, cloud-services-team (Kanban), PAWS
aborrero closed T251297: Refactor the toolforge::k8s::kubeadm* modules, a subtask of T211096: PAWS: Rebuild and upgrade Kubernetes, as Resolved.
Thu, May 7, 5:31 PM · Patch-For-Review, Toolforge, Epic, Goal, cloud-services-team (Kanban), PAWS
aborrero closed T251297: Refactor the toolforge::k8s::kubeadm* modules as Resolved.

This is done!

Thu, May 7, 5:31 PM · Toolforge, cloud-services-team (Kanban), PAWS
aborrero closed T250866: Stage packages for upstream kubeadm v1.16.9 to use in Toolforge, a subtask of T246122: Upgrade the Toolforge Kubernetes cluster to v1.16, as Resolved.
Thu, May 7, 5:30 PM · Patch-For-Review, Epic, Toolforge, cloud-services-team (Kanban), Kubernetes
aborrero closed T250866: Stage packages for upstream kubeadm v1.16.9 to use in Toolforge as Resolved.

This is now done.

Thu, May 7, 5:30 PM · Toolforge, cloud-services-team (Kanban), Kubernetes

Wed, May 6

aborrero triaged T232399: Improve https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS as Low priority.
Wed, May 6, 9:35 AM · User-srodlund, Toolforge, Documentation
aborrero added a comment to T232399: Improve https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS.

ok thanks!

Wed, May 6, 9:35 AM · User-srodlund, Toolforge, Documentation

Tue, May 5

aborrero closed T251660: Create prometheus-rabbitmq-exporter for buster, a subtask of T251294: Upgrade cloud-vps control plane to Debian Buster, as Resolved.
Tue, May 5, 10:24 AM · Patch-For-Review, cloud-services-team (Kanban)
aborrero closed T251660: Create prometheus-rabbitmq-exporter for buster as Resolved.

Unblocked! Please reopen if required.

Tue, May 5, 10:24 AM · cloud-services-team (Kanban)
aborrero closed T251575: Build prometheus-pdns-exporter for Buster, a subtask of T251294: Upgrade cloud-vps control plane to Debian Buster, as Resolved.
Tue, May 5, 10:21 AM · Patch-For-Review, cloud-services-team (Kanban)
aborrero closed T251575: Build prometheus-pdns-exporter for Buster as Resolved.

This should unblock this for now.

Tue, May 5, 10:21 AM · cloud-services-team (Kanban), observability
aborrero added a comment to T120225: Toolforge: correctly envelope forwarded email.

We usually use the toolsbeta project as the testing environment for tools (toolforge). That being said, I'm not sure if we would be able to test everything in toolsbeta (not sure if the email environment is the same or allows the same workflows that in tools). The domain there would be toolsbeta.wmflabs.org.

Tue, May 5, 9:56 AM · Patch-For-Review, Mail, Toolforge
aborrero renamed T120225: Toolforge: correctly envelope forwarded email from correctly envelope forwarded email to Toolforge: correctly envelope forwarded email.
Tue, May 5, 9:51 AM · Patch-For-Review, Mail, Toolforge

Mon, May 4

aborrero triaged T251660: Create prometheus-rabbitmq-exporter for buster as Medium priority.
Mon, May 4, 4:33 PM · cloud-services-team (Kanban)
aborrero claimed T251575: Build prometheus-pdns-exporter for Buster.
Mon, May 4, 4:33 PM · cloud-services-team (Kanban), observability
aborrero claimed T251660: Create prometheus-rabbitmq-exporter for buster.
Mon, May 4, 4:32 PM · cloud-services-team (Kanban)
aborrero added a comment to T251753: Request creation of wikidata-query-service-tutorial VPS project.

Yes, for simple websites Toolforge is a better place. But using wordpress inside Toolforge is not an easy task (if possible at all, I don't know). In fact, running content management systems in general is not well supported in Toolforge right now from my understanding. Among other things you don't have in Toolforge the level of database freedom that these kind of CMS usually require.

Mon, May 4, 2:58 PM · Cloud-VPS (Project-requests)

Wed, Apr 29

aborrero moved T250172: Toolforge: k8s: ingress: consider creating ingress-specific nodes from Needs discussion to Soon! on the cloud-services-team (Kanban) board.

On the 2020-04-29 WMCS meeting we decided this is something interesting to explore + using openstack server groups to ensure ingress nodes aren't in the same hypervisor.
We also agreed this is a low priority change, so we wont work on this in the short term?

Wed, Apr 29, 3:53 PM · cloud-services-team (Kanban), Tools
aborrero moved T251295: Plan the integration of new WMCS naming schemes into PAWS from Needs discussion to Soon! on the cloud-services-team (Kanban) board.

from the 2020-04-29 WMCS team meeting:

Wed, Apr 29, 3:45 PM · Toolforge, cloud-services-team (Kanban), PAWS

Apr 29 2020

aborrero added a comment to T250866: Stage packages for upstream kubeadm v1.16.9 to use in Toolforge.

We have explicit apt pins for these packages, in toolforge::k8s::kubeadmrepo.

Apr 29 2020, 12:41 PM · Toolforge, cloud-services-team (Kanban), Kubernetes
aborrero claimed T250866: Stage packages for upstream kubeadm v1.16.9 to use in Toolforge.
Apr 29 2020, 12:37 PM · Toolforge, cloud-services-team (Kanban), Kubernetes

Apr 28 2020

aborrero added a comment to T232399: Improve https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS.

Hi @srodlund, I would need to introduce a new page in wikitech for user help. It is related to best practices and how to use smtp (email) inside the CloudVPS service.

Apr 28 2020, 12:09 PM · User-srodlund, Toolforge, Documentation
aborrero added a comment to T235401: Emails from discourse-mediawiki.wmflabs.org softfail SPF.

Do we know if the software support the configuration change that @herron is suggesting? Could we try it regardless of fate of the project? (just for research/documentation/bugfixing purposes)

Apr 28 2020, 11:57 AM · Discourse

Apr 27 2020

aborrero added a comment to T249114: E-mails from noreply@pypi.org to tools.pywikibot@tools.wmflabs.org are not forwarded to certain recipients due to SPF.

@Dvorapa personal email server

is the no. 1 e-mail provider in Czech & Slovak Republics :D

Apr 27 2020, 2:55 PM · Pywikibot, Toolforge, cloud-services-team (Kanban)
aborrero added a comment to T249114: E-mails from noreply@pypi.org to tools.pywikibot@tools.wmflabs.org are not forwarded to certain recipients due to SPF.

@zhuyifei1999 in recent logs I see some emails being forwarded correctly. So indeed the problem is only with the pypi.org domain, right?

Apr 27 2020, 1:08 PM · Pywikibot, Toolforge, cloud-services-team (Kanban)

Apr 21 2020

aborrero added a comment to T250623: Allow providing a commit message for hieradata changes.

Bonus point: allow managing hieradata using git. This has the challenge of auth, proper multitenancy, etc, but worth noting anyway.

Apr 21 2020, 3:59 PM · Puppet, Horizon
aborrero closed T249837: Revamp the build process for debian packages in Toolforge as Resolved.

Summary of what was done here:

  • sbuild was introduced in our package builder VM
  • the wmcs-package-build script was created to automate most of our current workflow for building & distributing a package
  • some docs were created in wikitech
Apr 21 2020, 11:15 AM · Toolforge, cloud-services-team (Kanban)

Apr 20 2020

aborrero added a comment to T249837: Revamp the build process for debian packages in Toolforge.

I introduced the wmcs-package-build.py script and created https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Packaging

Apr 20 2020, 1:28 PM · Toolforge, cloud-services-team (Kanban)
aborrero awarded T250623: Allow providing a commit message for hieradata changes a Love token.
Apr 20 2020, 10:21 AM · Puppet, Horizon

Apr 16 2020

aborrero closed T250365: Unable to SSH into instances project 'wikilink' as Resolved.

Might be fixed now:

Apr 16 2020, 12:33 PM · Cloud-Services

Apr 15 2020

aborrero added a comment to T250206: Deploy a proof of concept prometheus server in cloudvps to replace shinken.

For the record, we have a grafana dashboard per project https://grafana-labs.wikimedia.org/d/000000059/cloud-vps-project-board?orgId=1&var-project=project-proxy&var-server=All&from=now-2d&to=now which is think is using data from graphite instead of prometheus.

Apr 15 2020, 9:47 AM · Patch-For-Review, Cloud-VPS (Debian Jessie Deprecation)
aborrero closed T135046: Whitelist labs instances that need XFF header passed through the web proxy as Resolved.

I spot checked the apache logs for the affected projects, and only the account project seems to dump the XFF header in the logs. In case you need it, they do something like:

Apr 15 2020, 9:43 AM · Privacy Engineering, cloud-services-team (Kanban), WMF-Legal, Privacy, Cloud-Services
aborrero added a comment to T135046: Whitelist labs instances that need XFF header passed through the web proxy.

Applied this change to hiera in horizon:

Apr 15 2020, 9:23 AM · Privacy Engineering, cloud-services-team (Kanban), WMF-Legal, Privacy, Cloud-Services
aborrero added a comment to T145703: Horizon loses credentials every day.

with the latest changes, I've been using horizon for 3 days in a row without having to enter my credentials! wow!

Apr 15 2020, 9:13 AM · Security, cloud-services-team (Kanban), Horizon

Apr 14 2020

aborrero added a comment to T135046: Whitelist labs instances that need XFF header passed through the web proxy.

@MusikAnimal your use case seems sensible. We can keep it whitelisted.

Apr 14 2020, 5:48 PM · Privacy Engineering, cloud-services-team (Kanban), WMF-Legal, Privacy, Cloud-Services
aborrero awarded T250206: Deploy a proof of concept prometheus server in cloudvps to replace shinken a Love token.
Apr 14 2020, 5:16 PM · Patch-For-Review, Cloud-VPS (Debian Jessie Deprecation)
aborrero removed a parent task for T215155: Toolforge: systemd monitoring: T236547: "shinken" Cloud VPS project jessie deprecation.
Apr 14 2020, 4:58 PM · cloud-services-team (Kanban), Toolforge
aborrero removed a subtask for T236547: "shinken" Cloud VPS project jessie deprecation: T215155: Toolforge: systemd monitoring.
Apr 14 2020, 4:58 PM · Cloud-VPS (Debian Jessie Deprecation)
aborrero added a parent task for T236547: "shinken" Cloud VPS project jessie deprecation: T194333: [Epic] Provide logging/metrics/monitoring SaaS for Cloud VPS tenants.
Apr 14 2020, 4:57 PM · Cloud-VPS (Debian Jessie Deprecation)
aborrero added a subtask for T194333: [Epic] Provide logging/metrics/monitoring SaaS for Cloud VPS tenants: T236547: "shinken" Cloud VPS project jessie deprecation.
Apr 14 2020, 4:57 PM · cloud-services-team (Kanban), Epic, Cloud-VPS
aborrero added a subtask for T194333: [Epic] Provide logging/metrics/monitoring SaaS for Cloud VPS tenants: T244809: Remove or fix stats collecting from tools-manifest (webservice-monitor).
Apr 14 2020, 4:56 PM · cloud-services-team (Kanban), Epic, Cloud-VPS
aborrero added a parent task for T244809: Remove or fix stats collecting from tools-manifest (webservice-monitor): T194333: [Epic] Provide logging/metrics/monitoring SaaS for Cloud VPS tenants.
Apr 14 2020, 4:56 PM · cloud-services-team (Kanban), Toolforge
aborrero added a parent task for T53434: Implement a system to monitor tools on tool-labs: T194333: [Epic] Provide logging/metrics/monitoring SaaS for Cloud VPS tenants.
Apr 14 2020, 4:55 PM · cloud-services-team (Kanban), User-Matthewrbowker, community-labs-monitoring, Toolforge
aborrero added a subtask for T194333: [Epic] Provide logging/metrics/monitoring SaaS for Cloud VPS tenants: T53434: Implement a system to monitor tools on tool-labs.
Apr 14 2020, 4:55 PM · cloud-services-team (Kanban), Epic, Cloud-VPS
aborrero added a parent task for T166845: monitor some things on all Cloud instances (discussion): T194333: [Epic] Provide logging/metrics/monitoring SaaS for Cloud VPS tenants.
Apr 14 2020, 4:54 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS
aborrero added a subtask for T194333: [Epic] Provide logging/metrics/monitoring SaaS for Cloud VPS tenants: T166845: monitor some things on all Cloud instances (discussion).
Apr 14 2020, 4:54 PM · cloud-services-team (Kanban), Epic, Cloud-VPS
aborrero triaged T250172: Toolforge: k8s: ingress: consider creating ingress-specific nodes as Lowest priority.
Apr 14 2020, 1:26 PM · cloud-services-team (Kanban), Tools
aborrero created T250172: Toolforge: k8s: ingress: consider creating ingress-specific nodes.
Apr 14 2020, 1:25 PM · cloud-services-team (Kanban), Tools
aborrero closed T249843: Toolforge.org: k8s canonical temporal redirect uses HTTP 301, a subtask of T234617: Toolforge. introduce new domain toolforge.org, as Resolved.
Apr 14 2020, 11:37 AM · Patch-For-Review, Goal, Toolforge, cloud-services-team (Kanban), Kubernetes
aborrero closed T249843: Toolforge.org: k8s canonical temporal redirect uses HTTP 301 as Resolved.
Apr 14 2020, 11:37 AM · Toolforge, cloud-services-team (Kanban), Kubernetes

Apr 13 2020

aborrero added a comment to T234617: Toolforge. introduce new domain toolforge.org.

There is interwiki prefix [[toollabs:...]] now, that still leads to old url. Will it be updated or will be introduced a new prefix?

Apr 13 2020, 12:48 PM · Patch-For-Review, Goal, Toolforge, cloud-services-team (Kanban), Kubernetes

Apr 10 2020

aborrero added a comment to T249837: Revamp the build process for debian packages in Toolforge.

I'm reviewing the current steps we have in our workflow:

Apr 10 2020, 11:11 AM · Toolforge, cloud-services-team (Kanban)
aborrero triaged T249843: Toolforge.org: k8s canonical temporal redirect uses HTTP 301 as High priority.

Patch is merged, we need another release of the tools-webservice package before we announce the toolforge.org domain next monday.

Apr 10 2020, 9:57 AM · Toolforge, cloud-services-team (Kanban), Kubernetes

Apr 9 2020

aborrero created T249843: Toolforge.org: k8s canonical temporal redirect uses HTTP 301.
Apr 9 2020, 4:26 PM · Toolforge, cloud-services-team (Kanban), Kubernetes
aborrero claimed T249837: Revamp the build process for debian packages in Toolforge.

Will take a look at this soon!

Apr 9 2020, 4:06 PM · Toolforge, cloud-services-team (Kanban)
aborrero created T249815: maps: whitelist/reduce ratelimit from requests with toolforge.org referrer .
Apr 9 2020, 1:16 PM · User-Urbanecm, Patch-For-Review, Operations, Maps, Toolforge, cloud-services-team (Kanban)
aborrero closed T219070: Failure of "kubectl get pods" for the editgroups project as Resolved.

Being a bit forward here today, I just bumped the nproc limit to 250 (from the previous value on 100 in the general bastion and 200 in the -dev one).
50 new procs per user should be more than enough to allow a couple of shell sessions opened in parallel and still honoring the shared/limited nature of our bastion servers.

Apr 9 2020, 11:22 AM · cloud-services-team (Kanban), Toolforge, Kubernetes
aborrero placed T248881: CloudVPS: research VXLAN implementation for neutron up for grabs.
Apr 9 2020, 11:02 AM · cloud-services-team (Kanban)
aborrero changed the status of T245606: CloudVPS: enable BGP in the neutron transport network, a subtask of T244727: CloudVPS: networking improvements, from Open to Stalled.
Apr 9 2020, 11:02 AM · cloud-services-team (Kanban), Epic
aborrero changed the status of T245606: CloudVPS: enable BGP in the neutron transport network from Open to Stalled.

We are not planning on working on this anytime soon.

Apr 9 2020, 11:02 AM · netops, cloud-services-team (Kanban), Operations