Dan Foy (dfoy) was off-boarded from production access and we need to check if he left data in his home dirs on stat*/notebook*/HDFS since he was part of:
- analytics-privatedata-users
Description
Description
Event Timeline
Comment Actions
====== stat1004 ====== total 1152 -rw-r--r-- 1 2318 wikidev 12564 Mar 24 2017 angola24.txt -rw-r--r-- 1 2318 wikidev 12836 Mar 27 2017 angola26.txt -rw-r--r-- 1 2318 wikidev 12555 Mar 29 2017 angola28.txt -rw-r--r-- 1 2318 wikidev 50965 Nov 14 2017 angola_nov_13_2017.txt -rw-r--r-- 1 2318 wikidev 12655 Nov 14 2017 angolanov2017.txt -rw-r--r-- 1 2318 wikidev 51064 Nov 14 2017 angola_nov_8_2017.txt -rw-r--r-- 1 2318 wikidev 50982 Nov 14 2017 angola_oct_13_2017.txt -rw-r--r-- 1 2318 wikidev 288 Nov 14 2017 angola.sql -rw-r--r-- 1 2318 wikidev 12694 Mar 24 2017 angola.txt -rw-r--r-- 1 2318 wikidev 2096 Mar 27 2017 asiacellfeb.txt -rw-r--r-- 1 2318 wikidev 389 Mar 14 2017 asiacellpv.sql -rw-r--r-- 1 2318 wikidev 56651 Apr 13 2017 asiacellunique.txt -rw-r--r-- 1 2318 wikidev 287 Mar 7 2017 gram2.sql -rw-r--r-- 1 2318 wikidev 305 Feb 1 2017 gram.sql -rw-r--r-- 1 2318 wikidev 6719 Jun 8 2017 iapv1to3.txt -rw-r--r-- 1 2318 wikidev 5173 Jun 8 2017 iapv4to6.txt -rw-r--r-- 1 2318 wikidev 0 Jun 8 2017 ia.txt -rw-r--r-- 1 2318 wikidev 12425 Jun 8 2017 ipv1to6.txt -rw-r--r-- 1 2318 wikidev 402 Jul 14 2017 iraqasiacell.sql -rw-r--r-- 1 2318 wikidev 4589 Jul 14 2017 iraqasiamayjune.txt -rw-r--r-- 1 2318 wikidev 2444 Feb 21 2017 iraqjan.txt -rw-r--r-- 1 2318 wikidev 4766 Jul 14 2017 iraqpv2.txt -rw-r--r-- 1 2318 wikidev 8057 Apr 13 2017 iraqpv.txt -rw-r--r-- 1 2318 wikidev 327 Jul 14 2017 iraq.sql -rw-r--r-- 1 2318 wikidev 1720 Jul 17 2017 iraquc1.txt -rw-r--r-- 1 2318 wikidev 0 Jul 14 2017 iraqundev2.txt -rw-r--r-- 1 2318 wikidev 4157 May 30 2017 iraqunicountrymay.txt -rw-r--r-- 1 2318 wikidev 112168 Jul 14 2017 iraquniqueasiacell2.txt -rw-r--r-- 1 2318 wikidev 85214 May 30 2017 iraquniqueasiacellmay.txt -rw-r--r-- 1 2318 wikidev 292 Mar 14 2017 iraquniqueasiacell.sql -rw-r--r-- 1 2318 wikidev 358 Jul 17 2017 iraquniquecountry.sql -rw-r--r-- 1 2318 wikidev 0 Jul 14 2017 iraquniquecountry.txt -rw-r--r-- 1 2318 wikidev 327 Mar 14 2017 iraquniquedevices.sql -rw-r--r-- 1 2318 wikidev 2850 Apr 13 2017 iraqunique.txt -rw-r--r-- 1 2318 wikidev 89394 Jun 8 2017 iuac.txt -rw-r--r-- 1 2318 wikidev 4402 Jun 8 2017 iun1to6.txt -rw-r--r-- 1 2318 wikidev 6719 Mar 31 2017 jan2marasiacellpv.txt -rw-r--r-- 1 2318 wikidev 2103 Mar 17 2017 marchext.txt -rw-r--r-- 1 2318 wikidev 1204 Mar 16 2017 march.txt -rw-r--r-- 1 2318 wikidev 391 Apr 24 2017 mexicopv.sql -rw-r--r-- 1 2318 wikidev 340 Apr 26 2017 mexico.sql -rw-r--r-- 1 2318 wikidev 402 Jun 27 2016 myanmar2016-2.sql -rw-r--r-- 1 2318 wikidev 415 May 3 2016 myanmar2016.sql -rw-r--r-- 1 2318 wikidev 305 Nov 2 2016 ncell2.sql -rw-r--r-- 1 2318 wikidev 335 Nov 2 2016 ncell.sql -rw-r--r-- 1 2318 wikidev 320 Oct 18 2016 ncell.sql.save -rw-r--r-- 1 2318 wikidev 340 Mar 29 2018 nigeria.sql -rw-rw-r-- 1 2318 wikidev 6940 Mar 29 2018 nigeria.txt -rw-r--r-- 1 2318 wikidev 2455 Jul 29 2016 outputargentina.txt -rw-r--r-- 1 2318 wikidev 7485 Aug 10 2016 outputmexico2.txt -rw-r--r-- 1 2318 wikidev 7485 Aug 10 2016 outputmexico3.txt -rw-r--r-- 1 2318 wikidev 7485 Aug 10 2016 outputmexico.txt -rw-r--r-- 1 2318 wikidev 4643 Jun 27 2016 outputmyanmar.txt -rw-r--r-- 1 2318 wikidev 10646 Oct 18 2016 outputncellnotwp2.txt -rw-r--r-- 1 2318 wikidev 37273 Nov 2 2016 outputncellnotwp2x.txt -rw-r--r-- 1 2318 wikidev 9090 Oct 18 2016 outputncellnotwp.txt -rw-r--r-- 1 2318 wikidev 2578 Oct 18 2016 outputncell.txt -rw-r--r-- 1 2318 wikidev 0 Jun 27 2016 outputserbia.txt -rw-r--r-- 1 2318 wikidev 6830 May 3 2016 output.txt -rw-r--r-- 1 2318 wikidev 300 May 8 2018 partnertoparticles.sql -rw-r--r-- 1 2318 wikidev 2477 Jun 8 2017 pvangolapt.txt -rw-r--r-- 1 2318 wikidev 316 Jun 8 2017 pvangola.sql -rw-r--r-- 1 2318 wikidev 2495 Jun 8 2017 pvangola.txt -rw-r--r-- 1 2318 wikidev 304 Mar 20 2018 pvindia.sql -rw-rw-r-- 1 2318 wikidev 53619 Mar 20 2018 pvindia.txt -rw-rw-r-- 1 2318 wikidev 54259 Mar 20 2018 pvindiayear.txt -rw-r--r-- 1 2318 wikidev 401 Jun 27 2016 serbia-2.sql -rw-r--r-- 1 2318 wikidev 428 Jul 24 2017 smilenigeriapv.sql -rw-r--r-- 1 2318 wikidev 8461 Jul 24 2017 smilenigeriapv.txt -rw-r--r-- 1 2318 wikidev 7055 Apr 26 2017 srilanka.txt -rw-r--r-- 1 2318 wikidev 14560 Mar 7 2017 tes2-19.txt -rw-r--r-- 1 2318 wikidev 389 Nov 20 2017 tigosenegal.sql -rw-r--r-- 1 2318 wikidev 12513 Nov 20 2017 tigo.txt -rw-r--r-- 1 2318 wikidev 339 Jun 8 2017 uniqueunitel2.sql -rw-r--r-- 1 2318 wikidev 292 Mar 29 2017 uniqueunitel.sql -rw-r--r-- 1 2318 wikidev 399 May 8 2018 unitelangola.sql -rw-r--r-- 1 2318 wikidev 4244 Mar 29 2017 unitelcommons.txt -rw-rw-r-- 1 2318 wikidev 2807 May 8 2018 uniteltest.txt -rw-r--r-- 1 2318 wikidev 54004 May 8 2018 unitel.txt -rw-r--r-- 1 2318 wikidev 858 Jun 8 2017 uunitel2.txt -rw-r--r-- 1 2318 wikidev 1908 Apr 24 2017 vivedigital5.txt -rw-r--r-- 1 2318 wikidev 0 Apr 24 2017 vivedigital6.txt -rw-r--r-- 1 2318 wikidev 0 Apr 24 2017 vivedigital.txt ls: cannot access '/var/userarchive/dfoy.tar.bz2': No such file or directory ====== stat1005 ====== total 0 ls: cannot access '/var/userarchive/dfoy.tar.bz2': No such file or directory ====== stat1006 ====== ls: cannot access '/srv/home/dfoy': No such file or directory ls: cannot access '/var/userarchive/dfoy.tar.bz2': No such file or directory ====== stat1007 ====== total 76 -rw-rw-r-- 1 2318 wikidev 343 Apr 6 2016 bangaldesh.sql -rw-rw-r-- 1 2318 wikidev 247 Feb 9 2016 bangladeshprojectjan.sql -rw-rw-r-- 1 2318 wikidev 312 Feb 8 2016 digi2.sql -rw-rw-r-- 1 2318 wikidev 276 Feb 5 2016 digi.sql -rw-rw-r-- 1 2318 wikidev 276 Feb 16 2016 djezzy.sql -rw-rw-r-- 1 2318 wikidev 314 Apr 6 2016 dtac.sql -rw-rw-r-- 1 2318 wikidev 275 Mar 16 2016 grameenphone2.sql -rw-rw-r-- 1 2318 wikidev 310 Apr 6 2016 grameenphonemar.sql -rw-rw-r-- 1 2318 wikidev 304 Feb 9 2016 grameenphone.sql -rw-rw-r-- 1 2318 wikidev 333 Apr 15 2016 mexicomonthly.sql -rw-rw-r-- 1 2318 wikidev 419 Apr 6 2016 montenegro2016.sql -rw-rw-r-- 1 2318 wikidev 419 May 3 2016 myanmar2016.sql -rw-rw-r-- 1 2318 wikidev 340 Apr 7 2016 myanmar.sql -rw-rw-r-- 1 2318 wikidev 362 Mar 28 2016 nepalopera.sql -rw-rw-r-- 1 2318 wikidev 278 Jan 15 2016 nepal.sql -rw-rw-r-- 1 2318 wikidev 295 Apr 28 2016 oneday.sql -rw-rw-r-- 1 2318 wikidev 0 May 3 2016 output.txt -rw-rw-r-- 1 2318 wikidev 419 Apr 6 2016 serbia2016.sql -rw-rw-r-- 1 2318 wikidev 341 Apr 6 2016 thailand.sql -rw-rw-r-- 1 2318 wikidev 261 Jan 30 2016 ukraine.sql ls: cannot access '/var/userarchive/dfoy.tar.bz2': No such file or directory ====== notebook1003 ====== total 0 ls: cannot access '/var/userarchive/dfoy.tar.bz2': No such file or directory ====== notebook1004 ====== total 4 drwxr-xr-x 7 2318 wikidev 4096 Jul 5 2018 venv ls: cannot access '/var/userarchive/dfoy.tar.bz2': No such file or directory ======= HDFS ======== Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 Found 1 items drwx------ - dfoy dfoy 0 2018-05-08 20:02 /user/dfoy/.staging
Comment Actions
@DFoy if you are still reading phab notifications, can you tell us if the above can be removed?
Comment Actions
I tracked down the other zero files I saved for @DFoy:
/wmf/data/archive/backup/zero-raw-logs-for-dan-foy
Let's ask @kzimmerman if anyone in PA is interested in analyzing this data, and delete if not. Kate - for context, these are old raw logs from the Wikipedia Zero (so they have IP / geo info)
Comment Actions
@elukey we have this data in the scrubbed, aggregated pageview data, correct? I recall there was a Wikipedia Zero tag. If so, then this should be deleted, I think.
Comment Actions
@kzimmerman, the aggregate data is available, yes, but Dan Foy wanted to analyze this for other reasons, it's much richer raw data. I'm not sure if he handed off those reasons during his off-boarding (doesn't seem like it). So that's why we're checking with you, just making sure before we delete potentially valuable data.
Comment Actions
@Milimetric nobody on Product Analytics is doing a dive into Wikipedia Zero; last time we dug into the data, the aggregates were sufficient for our needs.
I don't know what Dan was using this for. Have you or @elukey been in contact with Yael Weissberg (rweissburg@wikimedia.org), who was Dan's manager?
Comment Actions
@Milimetric let's pair together on this and decide if we can drop or not (when you have a moment).
Comment Actions
Everything cleaned up from stat/notebook homes. @Milimetric the last remaining action before closing is to remove or not the data from HDFS.
Comment Actions
Just removed /wmf/data/archive/backup/zero-raw-logs-for-dan-foy so this concludes this task.