Page MenuHomePhabricator

Upgrade all SWAP users to JupyterLab 1.0
Closed, ResolvedPublic

Description

It's possible for individual users to upgrade their JupyterLab version, but by default, users will remain on outdated versions.

JupyterLab recently released version 1.0 (release notes), and I've been using it for nearly a month without problems.

This seems like a good time to force upgrade everyone :)

Event Timeline

nshahquinn-wmf moved this task from Triage to Tracking on the Product-Analytics board.
elukey added a subscriber: elukey.Aug 19 2019, 3:28 PM

Hey Neil, is it something for us (as Analytics) to track or do you need any help/action from us? Not super clear what we'd need to do for JupyterLab..

elukey triaged this task as Medium priority.Aug 19 2019, 3:29 PM
elukey moved this task from Incoming to Project Newpyter on the Analytics board.
nshahquinn-wmf added a comment.EditedAug 19 2019, 3:42 PM

@elukey this is meant as something for y'all to do (I mentioned it in our last hangtime). The specific ask is upgrading the JupyterLab package to 1.0 within all SWAP virtualenvs; @Ottomata did a similar force upgrade in T198738 😁

Sure, I have no idea about jupyterlab so I'll have to dig a bit more into how it is installed/deployed/upgraded etc.. this is why I was asking :)

Ottomata claimed this task.Aug 26 2019, 2:24 PM

Change 532726 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[analytics/jupyterhub/deploy@master] Updating jupyterlab dependency to 1.0.9

https://gerrit.wikimedia.org/r/532726

Change 532726 merged by Ottomata:
[analytics/jupyterhub/deploy@master] Updating jupyterlab dependency to 1.0.9

https://gerrit.wikimedia.org/r/532726

Updated to JupyterLab 1.0.9 by following the process at https://wikitech.wikimedia.org/wiki/SWAP#Administration. I also manually upgraded this for all existing user venvs as described in the docs. JupyterLab users will start using the new version after they restart their JupyterHub servers.

Ottomata moved this task from Next Up to Done on the Analytics-Kanban board.

Change 532785 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[analytics/jupyterhub/deploy@master] Revert "Updating jupyterlab dependency to 1.0.9"

https://gerrit.wikimedia.org/r/532785

Change 532785 merged by Ottomata:
[analytics/jupyterhub/deploy@master] Revert "Updating jupyterlab dependency to 1.0.9"

https://gerrit.wikimedia.org/r/532785

Mentioned in SAL (#wikimedia-analytics) [2019-08-27T20:46:17Z] <ottomata> rolling back to jupyterlab version 0.32.1, 1.0.x is not compatible with Stretch's version of nodejs - T230724

After upgrading this morning, T231365: Trouble accessing Jupyter Lab (SWAP) was filed. I discovered that JupyterLab 1.0.x isn't compatible with the version of NodeJS that is available in Debian Stretch. Or at least, something really strange is going on with dependencies when building on Debian Stretch. I think perhaps that we have NodeJS locked to 6.11.0 via our WMF apt repository, and this is causing some strange dependency issues with npm when the JupyterLab wheel is being built.

If you can manually upgrade JupyterLab in your venv and it works, then go ahead! I think we need to sit on this old version until we either A. upgrade notebook servers to Buster (won't be a for a while due to T231067), or B. figure out how to containerize SWAP stuff (which is also a large unscheduled project).

Ottomata moved this task from Done to Paused on the Analytics-Kanban board.Aug 27 2019, 8:57 PM
Ottomata removed a project: Analytics-Kanban.

So, after more investigation, I think the reason my upgrade to 1.0.9 failed was because I was upgrading user venvs with the --ignore-installed flag instead of the --force-reinstall flag. This caused files from the other package version to be left around and used. I've just upgraded my JupyterLab back to 1.0.9 on notebook1004 and it seems fine.

I'd be willing to try again if I have some guinea pigs. @Iflorez & @Neil_P._Quinn_WMF perhaps?

Ottomata moved this task from Paused to In Code Review on the Analytics-Kanban board.

After upgrading on notebook1003, iflorez still encountered still had the same strange node dependency issues from yesterday. I don't have time to troubleshoot this now, so we'll have to leave this open.

After upgrading on notebook1003, iflorez still encountered still had the same strange node dependency issues from yesterday. I don't have time to troubleshoot this now, so we'll have to leave this open.

Thank you for working on this! I'm happy to be your guinea pig in the future if @Iflorez is unavailable.

As FYI stat100[5,8] (Buster nodes) are running with JupyterLab 1.1.0, if anybody wants to test and provide feedback I'd be happy :)

elukey claimed this task.Mar 27 2020, 5:16 PM
elukey added a project: User-Elukey.
elukey moved this task from Backlog to Done on the User-Elukey board.EditedApr 15 2020, 12:25 PM

If everybody agrees I'd call this task done. Migrating all users to jupyterlab 1.0 will be probably handled when we'll upgrade stat100[4,6,7] to Buster.

@nshahquinn-wmf what do you think?

nshahquinn-wmf closed this task as Resolved.Apr 19 2020, 4:51 PM

If everybody agrees I'd call this task done. Migrating all users to jupyterlab 1.0 will be probably handled when we'll upgrade stat100[4,6,7] to Buster.

@nshahquinn-wmf what do you think?

Seems reasonable! Thanks for your tireless labor in the server mines 😊