Page MenuHomePhabricator

bd808 (Bryan Davis)
Principal Software Engineer, Technical EngagementAdministrator

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Oct 3 2014, 2:36 PM (276 w, 3 d)
Roles
Administrator
Availability
Available
IRC Nick
bd808
LDAP User
BryanDavis
MediaWiki User
BDavis (WMF) [ Global Accounts ]

I'm BDavis (WMF) on wiki, bd808 on irc, and BryanDavis on Gerrit and Wikitech.

I've got a thing for ๐Ÿฆ„s. Don't judge.

I work for or provide services to the Wikimedia Foundation, but this is my only Phabricator account. Edits, statements, or other contributions made from this account are my own, and may not reflect the views of the Foundation.

Recent Activity

Fri, Jan 17

bd808 added a comment to T216020: Add python3 support for SULWatcher.

MySQLdb/ MySQL-python do not support Python 3. They were replaced by mysqlclient as far as I could read.

Fri, Jan 17, 9:56 PM ยท Patch-For-Review, Tool-stewardbots
bd808 added a project to T236446: Cloud Services shared IP (static NAT for external communications) often rate limited by YouTube for video downloads: Upstream.
Fri, Jan 17, 9:35 PM ยท Upstream, cloud-services-team (Kanban), Cloud-VPS, video2commons
bd808 added a comment to T243037: Shutdown scholarships.wikimedia.org and archive project.

@soworu It would be useful to check with Legal how we can make any use of the existing data. Knowing who has applied for scholarship in previous Wikimanias and who was accepted / declined may be useful information for scholarship committees in the current Wikimania and future events.

Fri, Jan 17, 4:26 PM ยท Patch-For-Review, Cleanup, Wikimedia-Wikimania-Scholarships
bd808 updated the task description for T210993: Deprecate Diamond collectors in Cloud VPS.
Fri, Jan 17, 4:23 PM ยท cloud-services-team (Kanban), User-fgiunchedi, observability, Operations
bd808 updated the task description for T236526: "design" Cloud VPS project jessie deprecation.
Fri, Jan 17, 3:38 AM ยท Cloud-VPS (Debian Jessie Deprecation)
bd808 added a comment to T236564: "wm-bot" Cloud VPS project jessie deprecation.

Pinged @Petrb on enwiki -- https://en.wikipedia.org/w/index.php?title=User_talk:Petrb&diff=936172441&oldid=926853139

Fri, Jan 17, 3:37 AM ยท Cloud-VPS (Debian Jessie Deprecation)
bd808 added a comment to T236577: "etytree" Cloud VPS project jessie deprecation.

@Epantaleo It looks like you were able to create an etytree-a.etytree.eqiad.wmflabs instance. Do you need further help to remove etytree-b.etytree.eqiad.wmflabs?

Fri, Jan 17, 3:34 AM ยท Cloud-VPS (Debian Jessie Deprecation)
bd808 added a comment to T236582: "automation-framework" Cloud VPS project jessie deprecation.

@crusnov Do you have a guess about when you might find time for this?

Fri, Jan 17, 3:29 AM ยท Cloud-VPS (Debian Jessie Deprecation)
bd808 updated the task description for T243037: Shutdown scholarships.wikimedia.org and archive project.
Fri, Jan 17, 12:47 AM ยท Patch-For-Review, Cleanup, Wikimedia-Wikimania-Scholarships
bd808 updated the task description for T243037: Shutdown scholarships.wikimedia.org and archive project.
Fri, Jan 17, 12:36 AM ยท Patch-For-Review, Cleanup, Wikimedia-Wikimania-Scholarships
bd808 updated the task description for T243037: Shutdown scholarships.wikimedia.org and archive project.
Fri, Jan 17, 12:10 AM ยท Patch-For-Review, Cleanup, Wikimedia-Wikimania-Scholarships

Thu, Jan 16

bd808 closed T86967: Convert Scholarships application to log directly to Logstash as Declined.

T243037: Shutdown scholarships.wikimedia.org and archive project

Thu, Jan 16, 11:45 PM ยท Wikimedia-Wikimania-Scholarships
bd808 closed T60070: Handle framework exceptions in Wikimania Scholarships application as Declined.

T243037: Shutdown scholarships.wikimedia.org and archive project

Thu, Jan 16, 11:44 PM ยท Wikimedia-Wikimania-Scholarships
bd808 closed T63269: CSRF warnings when users take a long time to fill out Wikimania Scholarships application as Declined.

T243037: Shutdown scholarships.wikimedia.org and archive project

Thu, Jan 16, 11:44 PM ยท Wikimedia-Wikimania-Scholarships
bd808 closed T152974: Create a means to separate links and other non-translated content from messages as Declined.
Thu, Jan 16, 11:44 PM ยท Wikimedia-Wikimania-Scholarships
bd808 added a comment to T152974: Create a means to separate links and other non-translated content from messages.

T243037: Shutdown scholarships.wikimedia.org and archive project

Thu, Jan 16, 11:44 PM ยท Wikimedia-Wikimania-Scholarships
bd808 closed T155674: Have list of wiki communities in Wikimania Scholarships app automatically update as Declined.

T243037: Shutdown scholarships.wikimedia.org and archive project

Thu, Jan 16, 11:43 PM ยท Wikimedia-Wikimania-Scholarships
bd808 closed T189559: Automate 'Phase 1' and 'Phase 3' of the review process of the Wikimania Scholarships App as Declined.

T243037: Shutdown scholarships.wikimedia.org and archive project

Thu, Jan 16, 11:43 PM ยท Outreach-Programs-Projects, Wikimedia-Wikimania-Scholarships
bd808 closed T119534: OAuth integration for applicants as Declined.

T243037: Shutdown scholarships.wikimedia.org and archive project

Thu, Jan 16, 11:42 PM ยท Patch-For-Review, Wikimedia-Wikimania-Scholarships
bd808 updated the task description for T243037: Shutdown scholarships.wikimedia.org and archive project.
Thu, Jan 16, 11:38 PM ยท Patch-For-Review, Cleanup, Wikimedia-Wikimania-Scholarships
bd808 moved T243037: Shutdown scholarships.wikimedia.org and archive project from Backlog to In Dev/Progress on the Wikimedia-Wikimania-Scholarships board.
Thu, Jan 16, 11:35 PM ยท Patch-For-Review, Cleanup, Wikimedia-Wikimania-Scholarships
bd808 updated the task description for T243037: Shutdown scholarships.wikimedia.org and archive project.
Thu, Jan 16, 11:35 PM ยท Patch-For-Review, Cleanup, Wikimedia-Wikimania-Scholarships
bd808 triaged T243037: Shutdown scholarships.wikimedia.org and archive project as Medium priority.
Thu, Jan 16, 11:32 PM ยท Patch-For-Review, Cleanup, Wikimedia-Wikimania-Scholarships
bd808 created T243037: Shutdown scholarships.wikimedia.org and archive project.
Thu, Jan 16, 11:32 PM ยท Patch-For-Review, Cleanup, Wikimedia-Wikimania-Scholarships
bd808 closed T236562: "wikidiff2-wmde-dev" Cloud VPS project jessie deprecation as Resolved.

Project deleted. Thanks @WMDE-Fisch

Thu, Jan 16, 11:11 PM ยท TCB-Team, WMDE-QWERTY-Team, WMDE-QWERTY-Sprint-2020-01-08, Cloud-VPS (Debian Jessie Deprecation)
bd808 claimed T243030: Slim application error when searching on quips.
Thu, Jan 16, 10:28 PM ยท Tools
bd808 added a comment to T242976: public dns for codfw1dev vms.

The agreed instance naming plan from https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/DNS_domain_usage#Resolution is to use *.{deployment}.wikimedia.cloud.

Thu, Jan 16, 10:26 PM ยท Epic, cloud-services-team (Kanban)
bd808 updated the task description for T242931: Improve mobile experience on Wikitech.
Thu, Jan 16, 8:23 PM ยท Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Mobile, wikitech.wikimedia.org, cloud-services-team
bd808 added a comment to T197930: Build or backport .deb for kubernetes python client for Stretch and Buster.

TLDR: I agree. Let's do that instead.

Thu, Jan 16, 8:19 PM ยท cloud-services-team (Kanban), Toolforge, Kubernetes
bd808 added a comment to T242719: https://tools.wmflabs.org/{toolname} no longer redirects to https://tools.wmflabs.org/{toolname}/ on new k8s cluster.

I manually added the annotation I suggested in T242719#5802258 to the Ingress object for extreg-wos (/usr/bin/kubectl edit ingress extreg-wos) and it appears to be doing the redirect as hoped. This should be a simple addition to the webservice generated object.

Default ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /extreg-wos/$2
  creationTimestamp: "2020-01-16T06:52:48Z"
  generation: 1
  labels:
    name: extreg-wos
    toolforge: tool
    tools.wmflabs.org/webservice: "true"
    tools.wmflabs.org/webservice-version: "1"
  name: extreg-wos
  namespace: tool-extreg-wos
  resourceVersion: "14574323"
  selfLink: /apis/extensions/v1beta1/namespaces/tool-extreg-wos/ingresses/extreg
-wos
  uid: f634d856-b986-4a03-9800-5d3c3e59084f
spec:
  rules:
  - host: tools.wmflabs.org
    http:
      paths:
      - backend:
          serviceName: extreg-wos
          servicePort: 8000
        path: /extreg-wos(/|$)(.*)
status:
  loadBalancer: {}
Edited ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      rewrite ^(/extreg-wos)$ $1/ redirect;
    nginx.ingress.kubernetes.io/rewrite-target: /extreg-wos/$2
  creationTimestamp: "2020-01-16T06:52:48Z"
  generation: 1
  labels:
    name: extreg-wos
    toolforge: tool
    tools.wmflabs.org/webservice: "true"
    tools.wmflabs.org/webservice-version: "1"
  name: extreg-wos
  namespace: tool-extreg-wos
  resourceVersion: "14576012"
  selfLink: /apis/extensions/v1beta1/namespaces/tool-extreg-wos/ingresses/extreg-wos
  uid: f634d856-b986-4a03-9800-5d3c3e59084f
spec:
  rules:
  - host: tools.wmflabs.org
    http:
      paths:
      - backend:
          serviceName: extreg-wos
          servicePort: 8000
        path: /extreg-wos(/|$)(.*)
status:
  loadBalancer: {}
Thu, Jan 16, 7:08 AM ยท Patch-For-Review, Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 added a comment to T242848: extreg-wos tool (Extension Registration Wall of Superpowers) broken (500 Internal Server Error).

I restarted it and it seems to work now. I looked for errors in $HOME/uwsgi.log before I restarted and did not see anything. I wonder if the Ingress object got messed up when @Legoktm or someone else was poking at T242719: https://tools.wmflabs.org/{toolname} no longer redirects to https://tools.wmflabs.org/{toolname}/ on new k8s cluster?

Thu, Jan 16, 6:57 AM ยท Toolforge
bd808 added a comment to T242931: Improve mobile experience on Wikitech.

@srodlund, I think the issue you mentioned about the Help namespace navboxes being stripped out on the mobile view is actually a "feature" of MobileFrontend. There is logic in that extension which includes removing all DOM elements which have a css class containing the string 'navbox'. We should be able to work around this by using a different class for the styling there, but we will have to think a bit about what we want that big pile of navigation to look like on smaller viewports. We probably do not want to make the start of each page be multiple vertical pages of the navigation content.

Thu, Jan 16, 6:42 AM ยท Readers-Web-Backlog (Tracking), Reading-Web-Local-Wiki-Issues, Mobile, wikitech.wikimedia.org, cloud-services-team
bd808 added a comment to T197930: Build or backport .deb for kubernetes python client for Stretch and Buster.

Do we actually need the client library for the relatively small number of things that webservice does with the Kubernetes API? Maybe it would be less effort to roll our own minimal client using requests? Webservice really only does these things with the API:

  • List Pods matching labels
  • List Deployments matching labels
  • List Services matching labels
  • List Ingresses matching labels
  • Create Deployment
  • Create Service
  • Create Ingress
  • Create Pod
  • Delete Ingress
  • Delete Service
  • Delete Deployment
  • Delete ReplicaSet
  • Delete Pod
Thu, Jan 16, 5:59 AM ยท cloud-services-team (Kanban), Toolforge, Kubernetes
bd808 updated StrikerBot.
Thu, Jan 16, 3:03 AM

Wed, Jan 15

bd808 added a comment to T242921: Tool "timerelengteam" loads resources from googleapis and cloudflare.

This is a really neat team tool by the way. :)

Wed, Jan 15, 10:32 PM ยท Privacy, Tools
bd808 created T242921: Tool "timerelengteam" loads resources from googleapis and cloudflare.
Wed, Jan 15, 10:31 PM ยท Privacy, Tools
bd808 moved T242187: "bigdisk" instance with "bigram" for Template Parameter Alignment Generation Scripts (Language team) from Inbox to Discussion needed on the Cloud-VPS (Quota-requests) board.
Wed, Jan 15, 9:53 PM ยท Cloud-VPS (Quota-requests), Language-Team (Language-2020-January-March)
bd808 added a comment to T242187: "bigdisk" instance with "bigram" for Template Parameter Alignment Generation Scripts (Language team).

@KartikMistry we discussed this request in our 2020-01-15 Cloud Services team meeting. One thing we wanted to know before making a decision is how "unique" the data you will be placing on this very large instance will be.

Wed, Jan 15, 9:41 PM ยท Cloud-VPS (Quota-requests), Language-Team (Language-2020-January-March)
bd808 closed T241584: Request increased quota for collection-alt-renderer Cloud VPS project as Declined.

Closing this round, but we should revisit this need and grant either NFS access or "better" storage to the project when it is ready to use it.

Wed, Jan 15, 8:27 PM ยท Cloud-VPS (Quota-requests)
bd808 added a comment to T242898: Mirroring Diffusion repositories to GitHub seems to be broken.

@mmodell Let me know if you need any superpowers on the GitHub side of this to debug things. I have the credentials for the https://github.com/toolforge-push account and admin level rights in the https://github.com/toolforge org account.

Wed, Jan 15, 7:29 PM ยท Diffusion, Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)), cloud-services-team (Kanban), Toolforge, Tools, Phabricator
bd808 moved T242898: Mirroring Diffusion repositories to GitHub seems to be broken from Inbox to Watching on the cloud-services-team (Kanban) board.
Wed, Jan 15, 6:36 PM ยท Diffusion, Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)), cloud-services-team (Kanban), Toolforge, Tools, Phabricator
bd808 created T242898: Mirroring Diffusion repositories to GitHub seems to be broken.
Wed, Jan 15, 6:36 PM ยท Diffusion, Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)), cloud-services-team (Kanban), Toolforge, Tools, Phabricator
bd808 added a comment to T236560: "dumps" Cloud VPS project jessie deprecation.

No response from either @Nemo_bis or @Hydriz yet. I just left a message on https://meta.wikimedia.org/w/index.php?title=User_talk:Nemo_bis&diff=19708450&oldid=19660923 to try another channel.

Wed, Jan 15, 6:20 PM ยท Cloud-VPS (Debian Jessie Deprecation)
bd808 closed T236548: "visualeditor" Cloud VPS project jessie deprecation as Resolved.

Thanks @Esanders. We are one step closer to world domination!! (Or maybe just getting rid of Jessie...)

Wed, Jan 15, 6:16 PM ยท Cloud-VPS (Debian Jessie Deprecation)
bd808 moved T242256: Request creation of ocrtoy VPS project from Inbox to Clinic Duty on the cloud-services-team (Kanban) board.
Wed, Jan 15, 4:19 PM ยท cloud-services-team (Kanban), Cloud-VPS (Project-requests)
bd808 added a project to T242256: Request creation of ocrtoy VPS project: cloud-services-team (Kanban).

+1

Wed, Jan 15, 4:19 PM ยท cloud-services-team (Kanban), Cloud-VPS (Project-requests)
bd808 moved T242812: Request creation of centralnotice-staging VPS project from Inbox to Clinic Duty on the cloud-services-team (Kanban) board.
Wed, Jan 15, 4:12 PM ยท cloud-services-team (Kanban), Cloud-VPS (Project-requests)
bd808 added a project to T242812: Request creation of centralnotice-staging VPS project: cloud-services-team (Kanban).

+1

Wed, Jan 15, 4:11 PM ยท cloud-services-team (Kanban), Cloud-VPS (Project-requests)
bd808 added a comment to T242187: "bigdisk" instance with "bigram" for Template Parameter Alignment Generation Scripts (Language team).

Any updates on this?

Wed, Jan 15, 5:45 AM ยท Cloud-VPS (Quota-requests), Language-Team (Language-2020-January-March)
bd808 merged task T239407: toolforge: new k8s: package newer/more convenient python3 k8s client libs into T197930: Build or backport .deb for kubernetes python client for Stretch and Buster.
Wed, Jan 15, 4:51 AM ยท Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 merged T239407: toolforge: new k8s: package newer/more convenient python3 k8s client libs into T197930: Build or backport .deb for kubernetes python client for Stretch and Buster.
Wed, Jan 15, 4:51 AM ยท cloud-services-team (Kanban), Toolforge, Kubernetes
bd808 removed a subtask for T228500: Toolforge: evaluate ingress mechanism: T234617: Toolforge. introduce new domain toolforge.org.
Wed, Jan 15, 4:49 AM ยท Patch-For-Review, Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 removed a parent task for T234617: Toolforge. introduce new domain toolforge.org: T228500: Toolforge: evaluate ingress mechanism.
Wed, Jan 15, 4:49 AM ยท Patch-For-Review, Goal, Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 added a comment to T197930: Build or backport .deb for kubernetes python client for Stretch and Buster.

Some thoughts on the current stated task goal:

  • webservice is currently python2 compatible
  • webservice is installed in all Toolforge Docker containers, many (most?) of which do not include python3
  • We have jessie containers still and no active plan to deprecate them (which we probably need to figure out soon?)
  • Python lovers are excited for the death of python2, but we will not have a python2-free distro until Bullseye which will be sometime in late 2021 (probably 2022 before it is active use in Toolforge)
Wed, Jan 15, 4:46 AM ยท cloud-services-team (Kanban), Toolforge, Kubernetes
bd808 updated the task description for T237784: Document migration plans and timelines.
Wed, Jan 15, 4:17 AM ยท Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 added a comment to T237784: Document migration plans and timelines.

https://wikitech.wikimedia.org/wiki/News/2020_Kubernetes_cluster_migration

Wed, Jan 15, 4:16 AM ยท Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 created T242824: Tool account cannot list all namespaced objects in its Kubernetes namespace.
Wed, Jan 15, 4:11 AM ยท Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 added a comment to T241584: Request increased quota for collection-alt-renderer Cloud VPS project.

NFS storage is Ok me.
[...snip...]
And of course it is Ok if you can provide the storage starting from August 2020, since I will need until then to prepare the PDFs anyway. And it also don't matter if you need a few more month.

Wed, Jan 15, 1:57 AM ยท Cloud-VPS (Quota-requests)

Tue, Jan 14

bd808 edited Description on Cloud-VPS (Quota-requests).
Tue, Jan 14, 11:28 PM
bd808 edited Description on Cloud-VPS (Quota-requests).
Tue, Jan 14, 11:28 PM
bd808 edited Description on Cloud-VPS (Project-requests).
Tue, Jan 14, 11:27 PM
bd808 committed R1958:0a4136cbf7f6: README: pedantic wording change (authored by bd808).
README: pedantic wording change
Tue, Jan 14, 10:41 PM
bd808 added a comment to T216020: Add python3 support for SULWatcher.

It looks like the bot is started on the job grid using jstart -N stewardbot -mem 2G /data/project/stewardbots/venv/bin/python2.7 /data/project/stewardbots/StewardBot/StewardBot.py, so the virtual environment that you need to run pip from is /data/project/stewardbots/venv. It is all python2 right now, so if you are going to try a python3 deployment you will need a new venv. Something like python3 -mvenv $HOME/venv-py3 would make you a new one.

Tue, Jan 14, 7:35 PM ยท Patch-For-Review, Tool-stewardbots
bd808 added a project to T234617: Toolforge. introduce new domain toolforge.org: Goal.
Tue, Jan 14, 5:51 PM ยท Patch-For-Review, Goal, Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 added a subtask for T229920: WMCS: migrate python2 scripts to python3: T241045: cloudmetrics: archive-instances using deprecated keystone lib.
Tue, Jan 14, 5:36 PM ยท Patch-For-Review, Epic, cloud-services-team (Kanban)
bd808 added a parent task for T241045: cloudmetrics: archive-instances using deprecated keystone lib: T229920: WMCS: migrate python2 scripts to python3.
Tue, Jan 14, 5:36 PM ยท cloud-services-team (Kanban)
bd808 placed T241045: cloudmetrics: archive-instances using deprecated keystone lib up for grabs.
Tue, Jan 14, 5:36 PM ยท cloud-services-team (Kanban)
bd808 assigned T242719: https://tools.wmflabs.org/{toolname} no longer redirects to https://tools.wmflabs.org/{toolname}/ on new k8s cluster to aborrero.
Tue, Jan 14, 5:23 PM ยท Patch-For-Review, Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 moved T242719: https://tools.wmflabs.org/{toolname} no longer redirects to https://tools.wmflabs.org/{toolname}/ on new k8s cluster from Important to Doing on the cloud-services-team (Kanban) board.
Tue, Jan 14, 5:23 PM ยท Patch-For-Review, Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 triaged T224510: Document the new NFS setup on cloudstore1008/9 as Medium priority.
Tue, Jan 14, 5:20 PM ยท cloud-services-team (Kanban)
bd808 closed T239415: Modify maintain-views to skip "#" on new dblist files as Resolved.
Tue, Jan 14, 5:14 PM ยท cloud-services-team (Kanban), Data-Services
bd808 added a comment to T241307: Improve inconsistent, long-winded, partially irrelevant, confusing [[mw:Gerrit/Tutorial]].
Tue, Jan 14, 5:08 PM ยท Developer-Advocacy (Jan-Mar 2020), Documentation
bd808 added a comment to T242719: https://tools.wmflabs.org/{toolname} no longer redirects to https://tools.wmflabs.org/{toolname}/ on new k8s cluster.

I apparently did not add this same logic to the fourohfour handler tool.

Would that work there? Seems like the annotation (which is another webservice thing, I imagine) might be the way to go on a quick reading.

Tue, Jan 14, 4:49 PM ยท Patch-For-Review, Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 added a comment to T242719: https://tools.wmflabs.org/{toolname} no longer redirects to https://tools.wmflabs.org/{toolname}/ on new k8s cluster.

The old redirect behavior comes from the admin tool when the front proxy redirects there for a 404:

if ( $notFoundHandler && $info['name'] !== false ) {
        // Route was for a known tool
        if ( $uri === "/{$info['name']}" ) {
                // Redirect bare /<toolname> to /<toolname>
                $this->redirect( "/{$info['name']}/", 301 );
        } else {
                // The tool's service must be down. Send a 503 response.
                $errorCode = '503';
        }
}

I apparently did not add this same logic to the fourohfour handler tool.

Tue, Jan 14, 4:26 PM ยท Patch-For-Review, Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 added a comment to T242719: https://tools.wmflabs.org/{toolname} no longer redirects to https://tools.wmflabs.org/{toolname}/ on new k8s cluster.

I think we could do this on the new cluster by adding an annotation to the Ingress object. I have not tried it yet, but something like nginx.ingress.kubernetes.io/configuration-snippet: rewrite ^(/TOOL_NAME)$ $1/ redirect; is suggested by https://github.com/kubernetes/ingress-nginx/issues/646.

Tue, Jan 14, 4:17 PM ยท Patch-For-Review, Toolforge, cloud-services-team (Kanban), Kubernetes
bd808 closed T242657: Find historical trends in Toolforge applications and approvals for future planning as Resolved.

@Bmueller please re-open and comment if this is not what you needed.

Tue, Jan 14, 5:16 AM ยท cloud-services-team (Kanban), Toolforge
bd808 added a comment to T242657: Find historical trends in Toolforge applications and approvals for future planning.

Based on the numbers from T242657#5800504, here are the year over year numbers we have for calendar years:

Tue, Jan 14, 5:03 AM ยท cloud-services-team (Kanban), Toolforge
bd808 added a comment to T242657: Find historical trends in Toolforge applications and approvals for future planning.

The last SQL step I used in T242657#5800494 to compute the percentage increase subtracted the contribution of following months, but did not also subtract the contribution of the current month. This made the resulting percentage the gross change rather than the rate of growth.

Tue, Jan 14, 4:41 AM ยท cloud-services-team (Kanban), Toolforge
bd808 added a comment to T242657: Find historical trends in Toolforge applications and approvals for future planning.

Translating the raw member additions into percentage increases against the total number of Toolforge members at a give date takes a bit of extra work. If this database was hosted on slightly newer version of MariaDB we could use the OVER window function to compute a running total. Because we don't have that function available, we will need to do a bit more work to compute running total. I decided to use temporary tables, but another approach that would scale better would be to write a python or php script to do the computations.

Tue, Jan 14, 4:34 AM ยท cloud-services-team (Kanban), Toolforge
bd808 added a comment to T242657: Find historical trends in Toolforge applications and approvals for future planning.

My original instinct was to figure this out by finding the Toolforge join date for each member from the LDAP directory and do some date based binning to show how this has changed over time. Unfortunately that method will not work because LDAP does not record group membership changes with per-member tracking--there is no "date added to group X" attribute on the user or the group.

Tue, Jan 14, 3:14 AM ยท cloud-services-team (Kanban), Toolforge
Krenair awarded T242642: Cleanup unsigned puppet client certs on tools-puppetmaster-01 a Evil Spooky Haunted Tree token.
Tue, Jan 14, 12:04 AM ยท Puppet, cloud-services-team (Kanban), Toolforge

Mon, Jan 13

bd808 closed T242538: Many grid engine backend webservices not registered at tools-proxy redis following depool restarts, a subtask of T242397: Make webservice grid jobs "non-rerunable", as Resolved.
Mon, Jan 13, 10:57 PM ยท Toolforge, cloud-services-team (Kanban)
bd808 closed T242538: Many grid engine backend webservices not registered at tools-proxy redis following depool restarts as Resolved.
$ comm -23 <(qstat -u \* -q webgrid-lighttpd -xml | grep JB_owner | grep -oP '(?<=<JB_owner>tools\.).+(?=</JB_owner>)' | sort) <(curl -s tools-proxy-05:8081/list | jq . | grep -oP '(?<=").+(?=": {)' | sort)
$
Mon, Jan 13, 10:57 PM ยท cloud-services-team (Kanban), Toolforge
bd808 added a comment to T242538: Many grid engine backend webservices not registered at tools-proxy redis following depool restarts.

After the scripted restarts, these 4 tools still had some issues:

  • convert
    • Job running on grid; no obvious errors in $HOME/error.log; an additional manual restart seems to have helped
  • dewikinews-rss
    • Job running on grid; $HOME/error.log so big that using it to debug is difficult (reported as T242680); manual stop/start seemed to bring service back up
  • omarghridabot
    • No sign that this tool was ever designed to run a webservice. The tool is showing up on the list because it has job named "replace.py" stuck in qw state because no queue name has been assigned. Possibly caused by a manual qalter replace.py command in the tool's $HOME/.bash_history?
  • sign-language-browser
    • Tool appears to be running as expected, but it is not listed in the curl -s tools-proxy-05:8081/list output. Another manual stop/start cycle fixed the proxy's tracking of the state.
Mon, Jan 13, 10:56 PM ยท cloud-services-team (Kanban), Toolforge
bd808 added a comment to T242680: dewikinews-rss: Huge $HOME/error.log output for each page load.

Reported upstream at https://github.com/dapete42/dewikinews-rss/issues/1 as well.

Mon, Jan 13, 10:21 PM ยท Tools
bd808 created T242680: dewikinews-rss: Huge $HOME/error.log output for each page load.
Mon, Jan 13, 10:17 PM ยท Tools
bd808 added a comment to T242538: Many grid engine backend webservices not registered at tools-proxy redis following depool restarts.

I ended up using this very magic command to perform the restarts and logging for most of the tools:

$ for tool in $(comm -23 <(qstat -u \* -q webgrid-lighttpd -xml | grep JB_owner | grep -oP '(?<=<JB_owner>tools\.).+(?=</JB_owner>)' | sort) <(curl -s tools-proxy-05:8081/list | jq . | grep -oP '(?<=").+(?=": {)' | sort)); do
echo $tool
sudo become $tool -- webservice restart
sudo become $tool -- dologmsg 'Restarted webservice to fix broken registration with the front proxy (T242538)'
done
catgraph
...........Restarting webservice............
cgstat
...........Restarting webservice.............
[...snip...]
zhdeletionpedia
..........Restarting webservice..............
zhwiki-qualifications-check
..........Restarting webservice..............
$

Thanks for that awesome live status comparison code @zhuyifei1999

Mon, Jan 13, 9:25 PM ยท cloud-services-team (Kanban), Toolforge
bd808 triaged T242538: Many grid engine backend webservices not registered at tools-proxy redis following depool restarts as Unbreak Now! priority.
Mon, Jan 13, 9:06 PM ยท cloud-services-team (Kanban), Toolforge
Bmueller awarded T242657: Find historical trends in Toolforge applications and approvals for future planning a Meh! token.
Mon, Jan 13, 8:59 PM ยท cloud-services-team (Kanban), Toolforge
bd808 created T242668: Tool "blockyquery" loads resources from cloudflare and jquery.com.
Mon, Jan 13, 8:45 PM ยท Toolforge-standards-committee, Tools, Privacy
bd808 added a comment to T205463: sudo randomly prompts for passwords in cloud instances when LDAP lookups fail.

I wonder if there is any way to set the equivalent of the -n, --non-interactive as the global default for sudo in all Cloud VPS instances? We could set up a global alias sudo=sudo -n which might help, but it would be more ideal if something can be added to /etc/sudoers that would make it just happen. The -n flag does not stop the "a password is required" error message, but it does disable the password prompt itself.

Mon, Jan 13, 8:38 PM ยท cloud-services-team (Kanban), Cloud-VPS, LDAP, Beta-Cluster-reproducible
bd808 created T242657: Find historical trends in Toolforge applications and approvals for future planning.
Mon, Jan 13, 7:00 PM ยท cloud-services-team (Kanban), Toolforge
bd808 claimed T242538: Many grid engine backend webservices not registered at tools-proxy redis following depool restarts.

Is it possible this is related to T242397: Make webservice grid jobs "non-rerunable" ?

Mon, Jan 13, 6:44 PM ยท cloud-services-team (Kanban), Toolforge
bd808 renamed T242538: Many grid engine backend webservices not registered at tools-proxy redis following depool restarts from Many webservice not registered at tools-proxy redis to Many grid engine backend webservices not registered at tools-proxy redis following depool restarts.
Mon, Jan 13, 6:42 PM ยท cloud-services-team (Kanban), Toolforge
bd808 added a subtask for T242397: Make webservice grid jobs "non-rerunable": T242538: Many grid engine backend webservices not registered at tools-proxy redis following depool restarts.
Mon, Jan 13, 6:41 PM ยท Toolforge, cloud-services-team (Kanban)
bd808 added a parent task for T242538: Many grid engine backend webservices not registered at tools-proxy redis following depool restarts: T242397: Make webservice grid jobs "non-rerunable".
Mon, Jan 13, 6:41 PM ยท cloud-services-team (Kanban), Toolforge
bd808 added a parent task for T242559: Partialy setup tools-k8s-worker instances created by novaadmin causing problems: T242632: Apparent issues in Toolforge Kubernetes.
Mon, Jan 13, 6:03 PM ยท cloud-services-team (Kanban), Toolforge
bd808 added a subtask for T242632: Apparent issues in Toolforge Kubernetes: T242559: Partialy setup tools-k8s-worker instances created by novaadmin causing problems.
Mon, Jan 13, 6:03 PM ยท cloud-services-team (Kanban), Toolforge
bd808 added a comment to T242632: Apparent issues in Toolforge Kubernetes.

There may be additional issues, but T242559: Partialy setup tools-k8s-worker instances created by novaadmin causing problems would have caused many problems for people and presented as random or nearly so with between 4 and 9 of the 2020 Kubernetes cluster nodes being in a state where $HOME would not have mounted into the pods.

Mon, Jan 13, 6:03 PM ยท cloud-services-team (Kanban), Toolforge
bd808 closed T242559: Partialy setup tools-k8s-worker instances created by novaadmin causing problems as Resolved.

I think this mess is cleaned up. T242637: Create a "health check" for Kubernetes worker nodes which validates local Toolforge config is my attempt at a task that would ideally catch this type of problem automatically instead of waiting for users to stumble across broken nodes and report them.

Mon, Jan 13, 5:54 PM ยท cloud-services-team (Kanban), Toolforge