Page MenuHomePhabricator

Trouble accessing Jupyter Lab (SWAP)
Closed, ResolvedPublic

Description

Having trouble this morning getting on JupyterLab.
I tried restarting the notebook, restarting my server connection, restarting my computer.
At present, I'm getting a blank screen when I navigate to JupyterLab.

I can access Jupyter Notebook...but, when I open the .pynb file that I was working with most recently, I am not able to see the code lines. I am only able to see the output.

Ideally, I would like to get back on JupyterLab and to get the files that I was working on, with the most recent saved changes.

Event Timeline

Iflorez created this task.Aug 27 2019, 5:42 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 27 2019, 5:42 PM

Do you mean in PAWS (in Cloud VPS), or SWAP, in Analytics Production?

I suspect you mean SWAP, since I just did T230724: Upgrade all SWAP users to JupyterLab 1.0 this morning.

Hello, yes, I mean SWAP. I will update this ticket to remove PAWS. Thank you.

Restricted Application added a project: Analytics. · View Herald TranscriptAug 27 2019, 6:15 PM

Something is definitely not working for me on notebook1003, am investigating. In the meantime, can you check if JupyterLab works for you on notebook1004?

no luck on notebook4, unfortunately

Ottomata moved this task from Next Up to Done on the Analytics-Kanban board.Aug 27 2019, 8:57 PM
Dzahn added a subscriber: Dzahn.Aug 28 2019, 9:24 AM

Icinga told us that the systemd state on both notebook1003 and notebook1004 was degraded.

Looking at the service that failed i saw:

● jupyter-iflorez-singleuser.service  loaded failed failed /bin/bash -c cd /home/iflorez && exec jupyterhub-singleuser --po
● jupyter-isaacj-singleuser.service   loaded failed failed /bin/bash -c cd /home/isaacj && exec jupyterhub-singleuser --por
● jupyter-nuria-singleuser.service    loaded failed failed /bin/bash -c cd /home/nuria && exec jupyterhub-singleuser --port
● jupyter-piccardi-singleuser.service loaded failed failed /bin/bash -c cd /home/piccardi && exec jupyterhub-singleuser --p
● jupyter-ryanmax-singleuser.service  loaded failed failed /bin/bash -c cd /home/ryanmax && exec jupyterhub-singleuser --po

I started the one for @Iflorez and it is running now. Did that fix it on notebook1003?

Mentioned in SAL (#wikimedia-operations) [2019-08-28T09:28:00Z] <mutante> notebook1004 - systemctl start jupyter-ebernhardson-singleuser (T231365)

Dzahn added a subscriber: EBernhardson.EditedAug 28 2019, 9:29 AM

on notebook1004 there was only 1 failed service, the one for @EBernhardson which i started. That made Icinga alerts recover.

On notebook1003 though starting the remaining failed services does not work. they keep failing.

https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=notebook1003&service=Check+systemd+state

Sorry, was talking with Iflorez in IRC last night and didn't update here. My upgrade (and downgrade) clearly had some problems. I'm working on this.

I think the issue had to do with the pip install command I was using to upgrade/rollback user venvs. I was using --ignore-installed instead of --force-reinstall. --ignore-installed will not remove old versions, just overwrite them. I think this was leaving old/bad package files in place.

Nuria closed this task as Resolved.Aug 29 2019, 5:06 PM
Aklapper renamed this task from Trouble accessing Jupyter Lab to Trouble accessing Jupyter Lab (SWAP).Sep 1 2019, 8:42 PM