Page MenuHomePhabricator
Create Task
Maniphest T343054

[bug] PAWS has problems loading
Closed, ResolvedPublic
Actions

Description

What happened?

PAWS ran smoothly until about an hour ago. Started with having a hard time getting the kernel to do anything, after which reloading resulted in 504 errors and blank screens.

I'm getting this error in the js console (and lots of similar errors):
Unsatisfied version 3.4.8 from @jupyterlab/application-top of shared singleton module @jupyterlab/apputils (required ^3.6.3)

Using Chrome on Mac. When I delete my stored data for wmcloud.org and am logged out, I do see the login screen and can log in. Immediately after logging in through mediawiki.org I get the same problems. Eventually it will load after several minutes, but then state
Server unavailable or unreachable
Your server at /user/Effeietsanders/ is not running. Would you like to restart it?

Related Objects
Search...

Event Timeline

Effeietsanders created this task.Jul 28 2023, 10:00 PM
Effeietsanders updated the task description. (Show Details)
Effeietsanders updated the task description. (Show Details)Jul 28 2023, 10:03 PM
Effeietsanders updated the task description. (Show Details)
Effeietsanders updated the task description. (Show Details)Jul 28 2023, 10:10 PM
Effeietsanders updated the task description. (Show Details)Jul 28 2023, 10:21 PM
Effeietsanders added a comment.Jul 28 2023, 10:51 PM

Found a workaround: stopping the server via the hub control panel, and then starting a server from the same resulted in a workable situation again. Not sure if this should be closed, or if there's an underlying issue that needs addressing.

rook subscribed.Jul 28 2023, 11:10 PM

This might be related to T338981. Jupyterlab is back versions at the moment as described in that ticket. Though keeping jupyterlab back is causing some limitations in other areas, this ticket being one. Let's keep this open for now.

rook added a subtask: T338981: [Rollback] paws to jupyterlab 4.0.Jul 28 2023, 11:10 PM
Effeietsanders added a comment.Jul 30 2023, 5:15 AM

@rook just fyi, I was having problems again today (between 9 and 10pm PT, if that helps) as a gradual slowdown and restarting kernel didn't help, but this was resolved after restarting the server.

rook added a comment.Jul 31 2023, 10:41 AM

Hi @Effeietsanders how long does it take for you to see a slowdown after you start your server? Or perhaps after you start working with the code you're using? Could you link me to the code that stops?

rook mentioned this in T343116: Upgrade paws helm chart.Jul 31 2023, 10:53 AM
Effeietsanders added a comment.Jul 31 2023, 5:24 PM

I have been running different versions of two notebooks (iterating to squeeze out bugs and data issues) this and this. I suspect the latter has been the most challenging - but it's hard to pin down as it only happened twice so far.

rook added a comment.Aug 1 2023, 8:12 PM

@Effeietsanders could you see if you can recreate the issue on hub-paws-dev.codfw1dev.wmcloud.org ?

Effeietsanders added a comment.Aug 1 2023, 11:25 PM

@rook I've been unable to reproduce even on the main server, but the hub-paws-dev actually doesn't even start up the server for me (it gets stuck at 2023-08-01T23:22:42Z [Normal] Pulling image "quay.io/wikimedia-paws-prod/singleuser:pr-317")

rook added a comment.Aug 2 2023, 10:44 AM
In T343054#9061325, @Effeietsanders wrote:

@rook I've been unable to reproduce even on the main server, but the hub-paws-dev actually doesn't even start up the server for me (it gets stuck at 2023-08-01T23:22:42Z [Normal] Pulling image "quay.io/wikimedia-paws-prod/singleuser:pr-317")

Oh I think that was because I forgot to do an initial login, so yours was the initial login. The singleuser container is big, so it usually takes longer than the timeout to get pulled down. Though the first login initiates the pull, so by the second login it is available. That said, I guess it doesn't make much difference what is happening in dev if the bug cannot be seen in production.

rook closed subtask T338981: [Rollback] paws to jupyterlab 4.0 as Resolved.Aug 15 2023, 2:58 PM
rook added a comment.Aug 30 2023, 11:03 AM

T343116 fixed some problems related to jupyterlab. I'm curious if this behavior can still be observed?

rook added a comment.Nov 6 2023, 8:07 PM

Sounds like this might have been resolved. Please re-open if not found to be the case.

rook closed this task as Resolved.Nov 6 2023, 8:07 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL