Page MenuHomePhabricator
Paste P5753

(An Untitled Masterwork)
ActivePublic

Authored by bd808 on Jul 17 2017, 1:37 AM.
Tags
None
Referenced Files
F8779520:
Jul 17 2017, 1:37 AM
Subscribers
None
$ sudo kubectl drain --delete-local-data --force tools-worker-1020.tools.eqiad.wmflabs
node "tools-worker-1020.tools.eqiad.wmflabs" cordoned
WARNING: Deleting pods with local storage: jupyter-haeb-27666631, jupyter-steak-12300809, jupyter-strainubot-7702059; Deleting pods not managed by ReplicationCo
ntroller, ReplicaSet, Job, or DaemonSet: jupyter-haeb-27666631, jupyter-steak-12300809, jupyter-strainubot-7702059
pod "enwnbot-1700914268-waq3z" deleted
pod "ifttt-4004504754-i5am6" deleted
pod "jupyter-haeb-27666631" deleted
pod "jupyter-steak-12300809" deleted
pod "jupyter-strainubot-7702059" deleted
node "tools-worker-1020.tools.eqiad.wmflabs" drained

Event Timeline

Before and after evacuation:

tools-worker-1020.tools:~
bd808$ df -h
Filesystem                                                Size  Used Avail Use%
Mounted on
udev                                                       10M     0   10M   0%
/dev
tmpfs                                                     1.6G  157M  1.5G  10%
/run
/dev/vda3                                                  19G   19G     0 100%
/
tmpfs                                                     4.0G     0  4.0G   0%
/dev/shm
tmpfs                                                     5.0M     0  5.0M   0%
/run/lock
tmpfs                                                     4.0G     0  4.0G   0%
/sys/fs/cgroup
labstore1003.eqiad.wmnet:/dumps                            28T   19T  9.1T  68%
/public/dumps
labstore1003.eqiad.wmnet:/scratch                         3.0T  1.3T  1.6T  46%
/mnt/nfs/labstore1003-scratch
nfs-tools-project.svc.eqiad.wmnet:/project/tools/home     8.0T  3.5T  4.1T  46%
/mnt/nfs/labstore-secondary-tools-home
nfs-tools-project.svc.eqiad.wmnet:/project/tools/project  8.0T  3.5T  4.1T  46%
/mnt/nfs/labstore-secondary-tools-project
tools-worker-1020.tools:~
bd808$ hostname -f
tools-worker-1020.tools.eqiad.wmflabs
tools-worker-1020.tools:~
bd808$ w
 01:38:40 up 17 days,  5:31,  1 user,  load average: 0.48, 0.76, 0.77
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
bd808    pts/0    bastion-01.basti 01:34    0.00s  0.14s  0.00s w
tools-worker-1020.tools:~
bd808$ df -h
Filesystem                                                Size  Used Avail Use% Mounted on
udev                                                       10M     0   10M   0% /dev
tmpfs                                                     1.6G  157M  1.5G  10% /run
/dev/vda3                                                  19G  2.8G   15G  16% /
tmpfs                                                     4.0G     0  4.0G   0% /dev/shm
tmpfs                                                     5.0M     0  5.0M   0% /run/lock
tmpfs                                                     4.0G     0  4.0G   0% /sys/fs/cgroup
labstore1003.eqiad.wmnet:/dumps                            28T   19T  9.1T  68% /public/dumps
labstore1003.eqiad.wmnet:/scratch                         3.0T  1.3T  1.6T  46% /mnt/nfs/labstore1003-scratch
nfs-tools-project.svc.eqiad.wmnet:/project/tools/home     8.0T  3.5T  4.1T  46% /mnt/nfs/labstore-secondary-tools-home
nfs-tools-project.svc.eqiad.wmnet:/project/tools/project  8.0T  3.5T  4.1T  46% /mnt/nfs/labstore-secondary-tools-project