Page MenuHomePhabricator

rook (Vivian Rook)
User

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Jun 7 2021, 2:32 AM (149 w, 1 d)
Availability
Available
LDAP User
Vivian Rook
MediaWiki User
VRook (WMF) [ Global Accounts ]

Recent Activity

Yesterday

rook closed T362102: Remove prometheus migrate logic as Resolved.
Mon, Apr 15, 2:48 PM · PAWS

Thu, Apr 11

rook closed T360233: store quarry state in object storage as Resolved.
Thu, Apr 11, 7:50 PM · Quarry
rook added a comment to T360233: store quarry state in object storage.

Docs in:
https://wikitech.wikimedia.org/wiki/Help:Object_storage_user_guide#S3_API
https://wikitech.wikimedia.org/wiki/Help:Using_OpenTofu_on_Cloud_VPS#State_management

Thu, Apr 11, 7:48 PM · Quarry

Wed, Apr 10

rook closed T362232: jupyterlab to 4.1.6 as Resolved.
Wed, Apr 10, 3:55 PM · PAWS
rook added a comment to T362213: Error 500 when clicking "stop query".

I'm guessing this issue is from the threads being in separate pods. The attached PR removes the feature which may be the way to go with it.

Wed, Apr 10, 3:03 PM · Quarry
rook created T362232: jupyterlab to 4.1.6.
Wed, Apr 10, 2:35 PM · PAWS

Tue, Apr 9

rook added a comment to T362157: Error in web instances..

Seems like this error is coming from the sqlite file not existing, all the requests seem to be for csv or json files. It is not clear who is calling for these files. This error should probably be caught as a file not found.

Tue, Apr 9, 2:32 PM · Quarry
rook created T362157: Error in web instances..
Tue, Apr 9, 12:40 PM · Quarry

Mon, Apr 8

rook added a comment to T362111: [bug] "Access denied" for quarry database user.

I've restarted the deployments. See how it behaves now?

Mon, Apr 8, 8:57 PM · Quarry
rook created T362102: Remove prometheus migrate logic.
Mon, Apr 8, 6:14 PM · PAWS
rook closed T361952: Reduce cluster size as Resolved.
Mon, Apr 8, 1:15 PM · PAWS

Fri, Apr 5

rook updated the task description for T361952: Reduce cluster size.
Fri, Apr 5, 3:40 PM · PAWS
rook created T361952: Reduce cluster size.
Fri, Apr 5, 3:39 PM · PAWS

Thu, Apr 4

rook created T361874: nova-compute proc minimum section in response page.
Thu, Apr 4, 7:38 PM · Cloud-VPS
rook created T361873: ProjectProxyMainProxyDown should have response page.
Thu, Apr 4, 7:36 PM · Cloud-VPS
rook closed T349031: Update helm for quarry on pr as Resolved.
Thu, Apr 4, 3:53 PM · Quarry
rook closed T349031: Update helm for quarry on pr, a subtask of T349029: Move quarry to magnum, as Resolved.
Thu, Apr 4, 3:52 PM · Quarry
rook added a comment to T361822: Lost saved queries.

I'm guessing these were lost during an upgrade to a newer version. Which was made from a backup of the db, suggesting that a backup would not recover them. I've been concerned that we would discover these kinds of issues on version migrations.

Thu, Apr 4, 2:41 PM · Regression, superset.wmcloud.org

Tue, Apr 2

rook created T361606: Remove superset-123-4 cluster.
Tue, Apr 2, 3:24 PM · superset.wmcloud.org
rook closed T361601: Upgrade to 3.1.1 as Resolved.
Tue, Apr 2, 3:21 PM · superset.wmcloud.org
rook created T361601: Upgrade to 3.1.1.
Tue, Apr 2, 2:27 PM · superset.wmcloud.org
rook closed T349029: Move quarry to magnum, a subtask of T349605: Remove quarry.wsgi on move to k8s, as Resolved.
Tue, Apr 2, 2:26 PM · Quarry
rook closed T349029: Move quarry to magnum as Resolved.
Tue, Apr 2, 2:26 PM · Quarry
rook closed T349029: Move quarry to magnum, a subtask of T349690: Move away from nfs?, as Resolved.
Tue, Apr 2, 2:26 PM · Quarry
rook closed T355707: Remove superset-123-3 cluster as Resolved.
Tue, Apr 2, 1:20 PM · superset.wmcloud.org
rook claimed T355707: Remove superset-123-3 cluster.
Tue, Apr 2, 1:07 PM · superset.wmcloud.org
rook closed T360916: Remove paws-123-12 cluster as Resolved.
Tue, Apr 2, 1:05 PM · PAWS
rook closed T361566: Request creation of o11y VPS project to replace monitoring as Resolved.
Tue, Apr 2, 12:25 PM · Cloud-VPS (Project-requests)

Mon, Apr 1

thcipriani awarded T361477: requests to increase quotas deployment-prep a Party Time token.
Mon, Apr 1, 9:12 PM · cloud-services-team, Cloud-VPS (Quota-requests), Beta-Cluster-Infrastructure
rook closed T361477: requests to increase quotas deployment-prep as Resolved.
Mon, Apr 1, 7:46 PM · cloud-services-team, Cloud-VPS (Quota-requests), Beta-Cluster-Infrastructure
rook added a comment to T361477: requests to increase quotas deployment-prep.

C'est fait

Mon, Apr 1, 7:46 PM · cloud-services-team, Cloud-VPS (Quota-requests), Beta-Cluster-Infrastructure
rook closed T361477: requests to increase quotas deployment-prep, a subtask of T327742: Migrate deployment-prep away from Debian Buster to Bullseye/Bookworm, as Resolved.
Mon, Apr 1, 7:45 PM · Cloud-VPS (Debian Buster Deprecation), Beta-Cluster-Infrastructure
rook closed T361506: Upgrade Rstudio as Resolved.
Mon, Apr 1, 6:37 PM · PAWS
rook created T361506: Upgrade Rstudio.
Mon, Apr 1, 6:12 PM · PAWS
rook added a comment to T361471: Quarry login fails due to redirect to plaintext HTTP URL.

I'm not sure how the backend saw it as https before. As the web proxy terminates tls and delivered to http://172.16.5.58:80. Though I suppose it must have delivered x-forwarded-proto along with that, and maybe k8s is altering that field.

Mon, Apr 1, 2:25 PM · Quarry
rook added a comment to T361471: Quarry login fails due to redirect to plaintext HTTP URL.

Regardless, I'm unsure of where that location header is populated from. There is no reason for it to be http.

Mon, Apr 1, 1:53 PM · Quarry
rook added a comment to T361471: Quarry login fails due to redirect to plaintext HTTP URL.

Strangely still seems to be letting log in with HTTPS-Only Mode enabled, restarted firefox just in case, still can log in

Mon, Apr 1, 1:52 PM · Quarry
rook added a comment to T361471: Quarry login fails due to redirect to plaintext HTTP URL.

I'm not able to reproduce this in firefox 124.0.1. Do I need to do anything additional to reproduce?

Mon, Apr 1, 1:39 PM · Quarry
rook added a parent task for T349032: Deploy magnum cluster for quarry: T361470: Shutdown quarry VMs.
Mon, Apr 1, 1:25 PM · Quarry
rook added a subtask for T361470: Shutdown quarry VMs: T349032: Deploy magnum cluster for quarry.
Mon, Apr 1, 1:25 PM · Quarry
rook created T361470: Shutdown quarry VMs.
Mon, Apr 1, 1:25 PM · Quarry
rook closed T349032: Deploy magnum cluster for quarry as Resolved.
Mon, Apr 1, 12:48 PM · Quarry
rook closed T349032: Deploy magnum cluster for quarry, a subtask of T360233: store quarry state in object storage, as Resolved.
Mon, Apr 1, 12:48 PM · Quarry
rook closed T349032: Deploy magnum cluster for quarry, a subtask of T349029: Move quarry to magnum, as Resolved.
Mon, Apr 1, 12:48 PM · Quarry
rook added a comment to T349032: Deploy magnum cluster for quarry.

Quarry is now on kubernetes.

Mon, Apr 1, 12:47 PM · Quarry
rook updated the task description for T349032: Deploy magnum cluster for quarry.
Mon, Apr 1, 12:05 PM · Quarry

Mon, Mar 25

rook closed T359591: add worker to paws as Resolved.
Mon, Mar 25, 4:32 PM · PAWS
rook moved T360690: Starting OpenRefine projects with Wikimedia Commons extension fails for some (but not all) users on PAWS from Backlog to Not PAWS on the PAWS board.
Mon, Mar 25, 4:31 PM · OpenRefine, PAWS
rook claimed T360916: Remove paws-123-12 cluster.
Mon, Mar 25, 4:14 PM · PAWS
rook created T360916: Remove paws-123-12 cluster.
Mon, Mar 25, 4:00 PM · PAWS
rook closed T360643: update helm chart and jupyterhub as Resolved.
Mon, Mar 25, 3:59 PM · PAWS
rook renamed T360800: Use upstream jupyter-rsession-proxy from Update jupyter-rsession-proxy to Use upstream jupyter-rsession-proxy.
Mon, Mar 25, 1:02 PM · PAWS
rook closed T360798: Update nb_serverproxy_openrefine as Resolved.
Mon, Mar 25, 12:39 PM · PAWS
rook added a comment to T360643: update helm chart and jupyterhub.

Github issue resolved. Try chart version 4.1.1

Mon, Mar 25, 12:32 PM · PAWS

Fri, Mar 22

rook closed T360803: PAWS partially down as Resolved.
Fri, Mar 22, 9:29 PM · PAWS
rook added a comment to T360803: PAWS partially down.

quay.io appears well again

Fri, Mar 22, 9:29 PM · PAWS
rook created T360803: PAWS partially down.
Fri, Mar 22, 5:08 PM · PAWS
rook updated the task description for T360800: Use upstream jupyter-rsession-proxy.
Fri, Mar 22, 4:59 PM · PAWS
rook created T360800: Use upstream jupyter-rsession-proxy.
Fri, Mar 22, 4:58 PM · PAWS
rook created T360798: Update nb_serverproxy_openrefine.
Fri, Mar 22, 4:57 PM · PAWS

Thu, Mar 21

rook added a comment to T360643: update helm chart and jupyterhub.

https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/3372

Thu, Mar 21, 7:52 PM · PAWS
rook added a comment to T360643: update helm chart and jupyterhub.

Getting

Not accepting cookie auth on POST /user/VRook%20%28WMF%29/rstudio/rpc/client_init: HTTP 403: Forbidden ('_xsrf' argument missing from POST)

When trying to open rstudio and openrefine

Thu, Mar 21, 6:57 PM · PAWS
rook created T360643: update helm chart and jupyterhub.
Thu, Mar 21, 2:58 PM · PAWS

Wed, Mar 20

rook created T360584: Remove redis from Quarry.
Wed, Mar 20, 11:15 PM · Quarry
rook closed T360539: Upgrade julia as Resolved.
Wed, Mar 20, 5:03 PM · PAWS
rook created T360539: Upgrade julia.
Wed, Mar 20, 4:06 PM · PAWS

Mar 15 2024

rook added a comment to T349032: Deploy magnum cluster for quarry.

I've opened T360233 to manage the tofu state in an object store. That way things shouldn't be stored locally, and tofu/deploy.sh can be run from wherever in the bastion.

Mar 15 2024, 8:59 PM · Quarry
rook added a parent task for T349032: Deploy magnum cluster for quarry: T360233: store quarry state in object storage.
Mar 15 2024, 8:57 PM · Quarry
rook added a subtask for T360233: store quarry state in object storage: T349032: Deploy magnum cluster for quarry.
Mar 15 2024, 8:57 PM · Quarry
rook created T360233: store quarry state in object storage.
Mar 15 2024, 8:57 PM · Quarry
rook added a comment to T349032: Deploy magnum cluster for quarry.

Oh there's a new k8s cluster for it. I've put the updated config in /opt/quarry-123-2.config

Mar 15 2024, 8:55 PM · Quarry
rook closed T360193: jupyterlab to 4.1.5 as Resolved.
Mar 15 2024, 4:16 PM · PAWS
rook created T360193: jupyterlab to 4.1.5.
Mar 15 2024, 2:23 PM · PAWS
rook added a comment to T349032: Deploy magnum cluster for quarry.

@SD0001 @Audiodude could yinz take a look at quarry-test.wmcloud.org and see if there are any obvious problems? The data is all a duplicate of the production quarry, though now everything is running in k8s (except for nfs and trove, which are also their own separate instance from the current production quarry) If it seems alright we can probably cut over next week.

Mar 15 2024, 1:17 PM · Quarry

Mar 13 2024

rook created T360041: Set query result retention time.
Mar 13 2024, 3:58 PM · Quarry

Mar 12 2024

rook closed T359924: remove unused image_name var as Resolved.
Mar 12 2024, 1:44 PM · PAWS
rook created T359924: remove unused image_name var.
Mar 12 2024, 11:52 AM · PAWS

Mar 9 2024

rook closed T359747: Increase paws nodes for outreachy demand as Resolved.
Mar 9 2024, 4:06 PM · PAWS
rook created T359747: Increase paws nodes for outreachy demand.
Mar 9 2024, 3:06 PM · PAWS

Mar 8 2024

rook closed T359616: New upstream release 9.0 for Pywikibot as Resolved.
Mar 8 2024, 12:33 PM · PAWS
rook created T359622: Move nfs off of puppet?.
Mar 8 2024, 12:05 PM · PAWS
rook closed T359588: jupyterlab to 4.1.4 as Resolved.
Mar 8 2024, 11:49 AM · PAWS

Mar 7 2024

rook created T359591: add worker to paws.
Mar 7 2024, 8:46 PM · PAWS
rook placed T358604: Update labpawspublic extension to jupyterlab 4 system up for grabs.
Mar 7 2024, 8:38 PM · PAWS
rook claimed T359588: jupyterlab to 4.1.4.
Mar 7 2024, 8:17 PM · PAWS
rook created T359588: jupyterlab to 4.1.4.
Mar 7 2024, 8:16 PM · PAWS
rook closed T327674: Remove puppet code related to paws kubeadmin cluster as Resolved.
Mar 7 2024, 8:15 PM · PAWS
rook added a comment to T327674: Remove puppet code related to paws kubeadmin cluster.

I believe this was removed with https://gerrit.wikimedia.org/r/c/operations/puppet/+/1006852

Mar 7 2024, 8:15 PM · PAWS
rook moved T341378: PAWS crash: User has exceeded the 'max_user_connections' resource (current value: 10) from Backlog to Not PAWS on the PAWS board.
Mar 7 2024, 8:12 PM · PAWS

Mar 6 2024

rook closed T358984: Add a simple script to connect to a replica database as Resolved.
Mar 6 2024, 3:12 PM · PAWS
rook closed T343676: PAWS shell - lack of i18n submodule or files or an outdated submodule as Declined.
Mar 6 2024, 12:11 PM · PAWS
rook added a comment to T343676: PAWS shell - lack of i18n submodule or files or an outdated submodule.

Closing, though please reopen if this is still an issue.

Mar 6 2024, 12:10 PM · PAWS
rook closed T359316: Increase paws nodes for outreachy demand as Resolved.
Mar 6 2024, 12:07 PM · PAWS
rook created T359316: Increase paws nodes for outreachy demand.
Mar 6 2024, 11:47 AM · PAWS

Mar 5 2024

rook closed T358621: remove jupyter-dash as Resolved.
Mar 5 2024, 1:26 PM · PAWS

Mar 4 2024

rook closed T358649: Add wikibase-cli to paws as Resolved.
Mar 4 2024, 1:58 PM · PAWS

Feb 27 2024

rook created T358621: remove jupyter-dash.
Feb 27 2024, 9:07 PM · PAWS
rook created T358604: Update labpawspublic extension to jupyterlab 4 system.
Feb 27 2024, 5:39 PM · PAWS

Feb 26 2024

rook added a comment to T358519: paws prometheus no longer 'trusted' in metricsinfra::alertmanager.

So far as I know the prometheus nodes that I removed in T356429 hadn't been doing anything for some time (A year or more).

Feb 26 2024, 7:22 PM · PAWS, Cloud-VPS, cloud-services-team
rook closed T312234: Jupyter command `jupyter-nbextension` not found. as Resolved.
Feb 26 2024, 4:10 PM · PAWS