Page MenuHomePhabricator

Check home/HDFS leftovers of effeietsanders
Closed, ResolvedPublic

Description

The access for Lodewijk Gelauff (effeietsanders) was removed. It needs to be checked if data was left in home dirs on stat*/HDFS since they were part of the "analytics-privatedata-users" group.

The Kerberos principal has already been removed. Point of contact wrt keeping data @MGerlach

Event Timeline

Change 821694 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] admin: remove access for effeietsanders

https://gerrit.wikimedia.org/r/821694

Change 821694 merged by Jbond:

[operations/puppet@production] admin: remove access for effeietsanders

https://gerrit.wikimedia.org/r/821694

MGerlach renamed this task from Check home/HDFS leftovers of aniketars to Check home/HDFS leftovers of effeietsanders.Aug 9 2022, 10:25 AM

@jbond Thanks. There should be data on stat1008. can we keep this, e.g. in my home folder somewhere?

Here is the listing of left over files.

btullis@marlin-wsl:~/bin$ ./user_leftovers.sh effeietsanders

====== stat1004 ======
total 0

====== stat1005 ======
total 12
-rw-r--r-- 1 effeietsanders wikidev 1546 Nov  1  2021 central_notice_analysis.ipynb
-rw-r--r-- 1 effeietsanders wikidev 5113 Nov  1  2021 Untitled.ipynb

====== stat1006 ======
total 0

====== stat1007 ======
total 0

====== stat1008 ======
total 60
drwxrwxrwx 8 effeietsanders wikidev  4096 Jul 18 02:11 shared_notebooks
-rwxr-xr-x 1 effeietsanders wikidev 50812 Nov  1  2021 test.ipnb
-rw-r--r-- 1 effeietsanders wikidev   576 Nov  1  2021 Untitled.ipynb

======= HDFS ========
Found 3 items
drwx------   - effeietsanders effeietsanders          0 2022-03-11 00:00 /user/effeietsanders/.Trash
drwxr-x---   - effeietsanders effeietsanders          0 2022-07-17 22:05 /user/effeietsanders/.sparkStaging
drwxr-x---   - effeietsanders effeietsanders          0 2022-01-09 00:44 /user/effeietsanders/output

====== Hive =========

I'm move those friles from stat1008 to a new subdirectory of your home directory @MGerlach

Can I take it that you don't want the files from stat1005?

Those files belonging to effeietsanders from stat1008 are now all available in /home/mgerlach/effeietsanders on the same host.

I have discovered that there are some files on HDFS as well, beneath these directories. Are these of any interest to you @MGerlach ?
I can provide a full file listing if you like.

btullis@an-launcher1002:~$ sudo -u hdfs kerberos-run-command hdfs hdfs dfs -ls /user/effeietsanders/output/csv
Found 5 items
drwxr-x---   - effeietsanders effeietsanders          0 2022-02-08 06:47 /user/effeietsanders/output/csv/agg_users
drwxr-x---   - effeietsanders effeietsanders          0 2022-01-09 05:08 /user/effeietsanders/output/csv/bi
drwxr-x---   - effeietsanders effeietsanders          0 2022-01-09 05:08 /user/effeietsanders/output/csv/ca
drwxr-x---   - effeietsanders effeietsanders          0 2022-01-09 05:09 /user/effeietsanders/output/csv/lp
drwxr-x---   - effeietsanders effeietsanders          0 2022-01-09 05:09 /user/effeietsanders/output/csv/up

@BTullis, thank you!

Those files belonging to effeietsanders from stat1008 are now all available in /home/mgerlach/effeietsanders on the same host.

Perfect. I can confirm that they are there and I can access.

Can I take it that you don't want the files from stat1005?

Correct. Files from stat1005 can be deleted.

I have discovered that there are some files on HDFS as well, beneath these directories. Are these of any interest to you @MGerlach ?

No. Files from HDFS can be deleted.

Thanks. All remaining files deleted from stat servers and HDFS.

Mentioned in SAL (#wikimedia-operations) [2022-09-04T12:48:58Z] <elukey> pkill remaining processes of user effeietsanders on stat1008 to unblock puppet - T314846