Page MenuHomePhabricator
Create Task
Maniphest T262847

Decomission SWAP
Closed, ResolvedPublic
Actions

Description

'Newypter' is currently installed only on stat1008. It is running alongside of the old SWAP virtualenv based jupyterhub instance. To finish up this project, we should

  • Update anaconda-wmf with new wmfdata release including this PR.
  • Clean up pyspark kernels
    • We don't want to support custom pyspark kernels anymore; users should use regular python kernels and load up spark themselves, probably via wmfdata. We need to add documentation on how to do this.
    • We can't actually remove these unless we know users aren't using them. Announcement should tell folks to stop, and we should remove these after we deprecate SWAP.
    • Make python spark work a little smarter with yarn and anaconda.
  • Re-Write good documentation in https://wikitech.wikimedia.org/wiki/Analytics/Systems/Jupyter for the new system: https://wikitech.wikimedia.org/wiki/User:Ottomata/Jupyter
  • Provision new jupyterhub setup on all stat boxes (this should just be applying the puppet class)
  • Make announcement asking users to switch to new conda based jupyterhub on a new port.
  • Set a date to decom SWAP.
  • Decom SWAP.
  • Remove all SWAP puppetization and documentation.

Details

SubjectRepoBranchLines +/-
Remove SWAP / virtualenv based jupyterhuboperations/puppetproduction+1 -524
Needed for new release of anaconda-wmf.operations/puppetproduction+2 -2
Include profile::analytics::jupyterhub on an-test-client1001operations/puppetproduction+5 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

Ottomata created this task.Sep 14 2020, 6:42 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 14 2020, 6:42 PM
Ottomata added a subscriber: • razzi.Sep 14 2020, 6:42 PM
mforns triaged this task as Medium priority.Sep 17 2020, 4:03 PM
mforns raised the priority of this task from Medium to High.
mforns moved this task from Incoming to Data Exploration Tools on the Analytics board.
mforns renamed this task from Prep for Newpyter promotion and SWAP decomission to Prep for replacing jupyter conda migration.Sep 17 2020, 4:06 PM
Ottomata updated the task description. (Show Details)Jan 13 2021, 5:07 PM
Ottomata updated the task description. (Show Details)Feb 26 2021, 9:43 PM
gerritbot added a comment.Feb 26 2021, 9:43 PM

Change 667276 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Include profile::analytics::jupyterhub on an-test-client1001

https://gerrit.wikimedia.org/r/667276

gerritbot added a project: Patch-For-Review.Feb 26 2021, 9:43 PM
gerritbot added a comment.Feb 27 2021, 12:55 AM

Change 667276 merged by Ottomata:
[operations/puppet@production] Include profile::analytics::jupyterhub on an-test-client1001

https://gerrit.wikimedia.org/r/667276

gerritbot added a comment.Mar 17 2021, 7:37 PM

Change 673084 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Needed for new release of anaconda-wmf.

https://gerrit.wikimedia.org/r/673084

Ottomata updated the task description. (Show Details)Mar 17 2021, 7:40 PM
Stashbot added a comment.Mar 17 2021, 8:15 PM

Mentioned in SAL (#wikimedia-analytics) [2021-03-17T20:15:11Z] <ottomata> install anaconda-wmf 2020.02~wmf3 on analytics cluster clients and workers - T262847

gerritbot added a comment.Mar 17 2021, 8:27 PM

Change 673084 merged by Ottomata:
[operations/puppet@production] Needed for new release of anaconda-wmf.

https://gerrit.wikimedia.org/r/673084

Ottomata updated the task description. (Show Details)Mar 18 2021, 3:04 PM
Ottomata closed subtask T271960: New anaconda-wmf release with updated packages as Resolved.Apr 6 2021, 7:12 PM
Ottomata renamed this task from Prep for replacing jupyter conda migration to Decomission SWAP.Apr 30 2021, 2:20 PM
Ottomata claimed this task.
Ottomata added a project: Analytics-Kanban.
Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.
Ottomata updated the task description. (Show Details)
gerritbot added a comment.Apr 30 2021, 3:31 PM

Change 683916 had a related patch set uploaded (by Ottomata; author: Ottomata):

[operations/puppet@production] Remove SWAP / virtualenv based jupyterhub

https://gerrit.wikimedia.org/r/683916

Stashbot added a comment.May 3 2021, 2:23 PM

Mentioned in SAL (#wikimedia-analytics) [2021-05-03T14:23:43Z] <ottomata> stopping all venv based jupyter singleuser servers - T262847

gerritbot added a comment.May 3 2021, 2:27 PM

Change 683916 merged by Ottomata:

[operations/puppet@production] Remove SWAP / virtualenv based jupyterhub

https://gerrit.wikimedia.org/r/683916

Ottomata added a comment.May 3 2021, 2:45 PM

Did the following on each stat box:

# find all jupyterhub singeluser processes using a venv and stop them.
for u in $(ps aux | grep  -E 'jupyterhub.*singleuser' | grep venv | awk '{print $1}'); do echo "Stopping jupyter-${u}-singleuser"; sudo systemctl stop jupyter-${u}-singleuser; done

sudo systemctl reset-failed

# Stop venv based jupyterhub
sudo systemctl stop jupyterhub

# remove all venv jupyterhub related files
sudo rm -rfv /lib/system/systemd/jupyterhub.service /etc/jupyterhub /var/lib/jupyterhub /var/log/jupyterhub /srv/jupyterhub /usr/local/share/jupyter
Ottomata updated the task description. (Show Details)May 3 2021, 2:46 PM
Ottomata moved this task from In Progress to Done on the Analytics-Kanban board.May 3 2021, 2:48 PM
Ottomata closed this task as Resolved.May 18 2021, 4:54 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits