Page MenuHomePhabricator

Check home/HDFS leftovers of shiladsen
Closed, ResolvedPublic

Description

The access for Shilad Sen (shiladsen) was removed. It needs to be checked if data was left in home dirs on stat*/notebook*/HDFS since they were part of the "analytics-privatedata-users" group.

There was no Kerberos principal for this user.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2020, 7:13 AM
====== stat1004 ======
total 7078300
drwxr-xr-x 15 shiladsen wikidev       4096 Dec 29  2017 analytics-refinery
drwxrwxr-x  7 shiladsen wikidev       4096 Nov 28  2017 annoy-master
-rw-r--r--  1 shiladsen wikidev     678187 Nov 28  2017 annoy-master.zip
-rw-rw-r--  1 shiladsen wikidev        694 Dec 26  2017 Available.class
-rw-rw-r--  1 shiladsen wikidev        277 Dec 26  2017 Available.java
drwxrwxr-x  2 shiladsen wikidev      61440 Dec 22  2017 corpus
drwxrwxr-x  9 shiladsen wikidev       4096 Dec 31  2017 fastText
-rw-rw-r--  1 shiladsen wikidev  209715200 Dec 23  2017 foo.txt
-rwxrwxr-x  1 shiladsen wikidev       1964 Mar 20  2018 ft.sh
-rwxr-xr-x  1 shiladsen wikidev    1595408 Nov 28  2017 get-pip.py
-rw-rw-r--  1 shiladsen wikidev      10930 Dec  2  2017 out.txt
drwxr-xr-x  2 shiladsen wikidev       4096 Nov  8  2017 pop
-rw-r--r--  1 shiladsen wikidev     489406 Nov 28  2017 setuptools-38.2.3-py2.py3-none-any.whl
drwxr-xr-x  3 shiladsen wikidev       4096 Mar 23  2018 shilad
-rw-rw-r--  1 shiladsen wikidev 2570437622 Mar  9  2018 sitelinks.tsv.bz2
-rw-rw-r--  1 shiladsen wikidev 4465019241 Jan 27  2018 vec_i10_z200_w7_s1e3_n10.txt.bz2
-rwxrwxr-x  1 shiladsen wikidev       2029 Jan 27  2018 w2v.sh
drwxrwxr-x  6 shiladsen wikidev       4096 May 13  2017 word2vec-master
-rw-r--r--  1 shiladsen wikidev     108059 Nov 28  2017 word2vec-master.zip
ls: cannot access '/var/userarchive/shiladsen.tar.bz2': No such file or directory

====== stat1005 ======
total 0
ls: cannot access '/var/userarchive/shiladsen.tar.bz2': No such file or directory

====== stat1006 ======
total 0
ls: cannot access '/var/userarchive/shiladsen.tar.bz2': No such file or directory

====== stat1007 ======
total 3025212
drwxr-xr-x 12 10339 wikidev       4096 May 24  2019 analytics-refinery
drwxr-xr-x 12 10339 wikidev       4096 Jun 14  2019 analytics-source
-rw-rw-r--  1 10339 wikidev  181675417 Nov 27  2017 counts.txt
-rw-rw-r--  1 10339 wikidev          0 Dec  2  2017 out.txt
-rw-r--r--  1 10339 wikidev     227545 May 25  2019 part-00000
-rw-rw-r--  1 10339 wikidev  293876402 Dec 29  2017 screenlog.0
-rw-rw-r--  1 10339 wikidev 2622014398 Dec 19  2017 sitelinks.tsv
ls: cannot access '/var/userarchive/shiladsen.tar.bz2': No such file or directory

====== stat1008 ======
total 0
ls: cannot access '/var/userarchive/shiladsen.tar.bz2': No such file or directory

======= HDFS ========
Found 8 items
drwxr-xr-x   - shiladsen shiladsen           0 2019-06-14 03:00 /user/shiladsen/.sparkStaging
drwx------   - shiladsen shiladsen           0 2019-06-14 03:38 /user/shiladsen/.staging
drwxr-xr-x   - shiladsen shiladsen           0 2017-11-27 19:10 /user/shiladsen/corpus
drwxr-xr-x   - shiladsen shiladsen           0 2019-05-25 04:43 /user/shiladsen/foo.txt
drwxr-xr-x   - shiladsen shiladsen           0 2018-03-09 19:07 /user/shiladsen/sitelink-export
drwxr-xr-x   - shiladsen shiladsen           0 2019-06-14 03:12 /user/shiladsen/sitelinks
drwxr-xr-x   - shiladsen shiladsen           0 2019-06-14 02:45 /user/shiladsen/sitelinks2
-rw-r--r--   3 shiladsen shiladsen 38086296251 2019-05-23 20:28 /user/shiladsen/wikidata-20190520-all.json.bz2

====== Hive =========
drwxrwxrwt   - shiladsen        hadoop                          0 2017-12-19 22:17 /user/hive/warehouse/shilad.db/raw_sitelinks
drwxrwxrwt   - shiladsen        hadoop                          0 2017-12-30 04:46 /user/hive/warehouse/shilad.db/sessions

@Shilad anything that we need to keep?

razzi moved this task from Incoming to Ops Week on the Analytics board.Oct 15 2020, 4:01 PM

You can trash everything! Sorry for the delayed response.

Deleted all the home dirs on stat100x, only hdfs files are left :)

I deleted HDFS and HIVE files.
Resolving!

mforns closed this task as Resolved.Oct 23 2020, 10:27 AM
mforns claimed this task.