project for dumps (semi-missing files)
Closed, InvalidPublic
Actions

Assigned To

None

Authored By

	Nemo_bis
	Jan 20 2015, 8:30 AM

Description

I'm pretty sure that 1) ls should work, 2) "?????????? ? ? ? ?" can't be good output for ls, 3) 4.0 KB for everything smells.

nemobis@dumps-stats:/data/project$ ls -lh wikistats/
ls: cannot access wikistats/dammit.lt: Permission denied
ls: cannot access wikistats/analytics: Permission denied
ls: cannot access wikistats/viz_gallery: Permission denied
ls: cannot access wikistats/dumps.old.tmp: Permission denied
ls: cannot access wikistats/squids: Permission denied
ls: cannot access wikistats/tmp: Permission denied
ls: cannot access wikistats/pediapress: Permission denied
ls: cannot access wikistats/mail-lists: Permission denied
ls: cannot access wikistats/_git_sandbox: Permission denied
ls: cannot access wikistats/backup: Permission denied
ls: cannot access wikistats/reportcard: Permission denied
ls: cannot access wikistats/lib: Permission denied
ls: cannot access wikistats/dumps: Permission denied
ls: cannot access wikistats/progress: Permission denied
ls: cannot access wikistats/animations: Permission denied
ls: cannot access wikistats/MetricsReporting: Permission denied
ls: cannot access wikistats/portal: Permission denied
ls: cannot access wikistats/pageviews_reports: Permission denied
total 0
?????????? ? ? ? ?            ? analytics
?????????? ? ? ? ?            ? animations
?????????? ? ? ? ?            ? backup
?????????? ? ? ? ?            ? dammit.lt
?????????? ? ? ? ?            ? dumps
?????????? ? ? ? ?            ? dumps.old.tmp
?????????? ? ? ? ?            ? _git_sandbox
?????????? ? ? ? ?            ? lib
?????????? ? ? ? ?            ? mail-lists
?????????? ? ? ? ?            ? MetricsReporting
?????????? ? ? ? ?            ? pageviews_reports
?????????? ? ? ? ?            ? pediapress
?????????? ? ? ? ?            ? portal
?????????? ? ? ? ?            ? progress
?????????? ? ? ? ?            ? reportcard
?????????? ? ? ? ?            ? squids
?????????? ? ? ? ?            ? tmp
?????????? ? ? ? ?            ? viz_gallery
nemobis@dumps-stats:/data/project$ sudo ls -lh wikistats/
total 72K
drwxr-xr-x  7 nemobis wikidev 4.0K Jun 19  2013 analytics
drwxr-xr-x  4 nemobis wikidev 4.0K Jun 19  2013 animations
drwxr-xr-x  2 nemobis wikidev 4.0K Jun 19  2013 backup
drwxr-xr-x  4 nemobis wikidev 4.0K Jun 19  2013 dammit.lt
drwxr-xr-x  8 nemobis wikidev 4.0K Mar 12  2014 dumps
drwxr-xr-x  3 nemobis wikidev 4.0K Mar 12  2014 dumps.old.tmp
drwxr-xr-x  4 nemobis wikidev 4.0K Mar 12  2014 _git_sandbox
drwxr-xr-x  2 nemobis wikidev 4.0K Mar 12  2014 lib
drwxr-xr-x  3 nemobis wikidev 4.0K Mar 12  2014 mail-lists
drwxr-xr-x  5 nemobis wikidev 4.0K Jun 19  2013 MetricsReporting
drwxr-xr-x  8 nemobis wikidev 4.0K Mar 12  2014 pageviews_reports
drwxr-xr-x  2 nemobis wikidev 4.0K Mar 12  2014 pediapress
drwxr-xr-x  3 nemobis wikidev 4.0K Mar 12  2014 portal
drwxr-xr-x  3 nemobis wikidev 4.0K Mar 12  2014 progress
drwxr-xr-x  3 nemobis wikidev 4.0K Mar 12  2014 reportcard
drwxr-xr-x 19 nemobis wikidev 4.0K Mar 12  2014 squids
drwxr-xr-x  2 nemobis wikidev 4.0K Jun 19  2013 tmp
drwxr-xr-x  2 nemobis wikidev 4.0K Jun 19  2013 viz_gallery
nemobis@dumps-stats:/data/project$  ^C
nemobis@dumps-stats:/data/project$ sudo stat wikistats/analytics
  File: ‘wikistats/analytics’
  Size: 4096            Blocks: 8          IO Block: 1048576 directory
Device: 1ch/28d Inode: 159678811   Links: 7
Access: (0755/drwxr-xr-x)  Uid: ( 2179/ nemobis)   Gid: (  500/ wikidev)
Access: 2014-03-10 03:50:42.089552626 +0000
Modify: 2013-06-19 06:26:35.000000000 +0000
Change: 2014-03-10 03:50:42.089552626 +0000
 Birth: -
nemobis@dumps-stats:/data/project$ df
Filesystem                                     1K-blocks        Used   Available Use% Mounted on
/dev/vda1                                        7742360     1088364     6237664  15% /
none                                                   4           0           4   0% /sys/fs/cgroup
udev                                             8212684           8     8212676   1% /dev
tmpfs                                            1643396        1092     1642304   1% /run
none                                                5120           0        5120   0% /run/lock
none                                             8216960           0     8216960   0% /run/shm
none                                              102400           0      102400   0% /run/user
/dev/vda2                                        1935632      794936     1024320  44% /var
labstore.svc.eqiad.wmnet:/scratch             7750208512  2339965952  5410242560  31% /data/scratch
labstore.svc.eqiad.wmnet:/project/dumps/home 32083929088 31155326976   928602112  98% /home
labstore.svc.eqiad.wmnet:/keys                    983040       46080      936960   5% /public/keys
labstore1003.eqiad.wmnet:/dumps              46864901120 10298499072 36566385664  22% /public/dumps
labstore.svc.eqiad.wmnet:/backups               19092480     1665024    16450560  10% /public/backups

Event Timeline

Nemo_bis created this task.Jan 20 2015, 8:30 AM

Nemo_bis raised the priority of this task from to Needs Triage.

Nemo_bis updated the task description. (Show Details)

Nemo_bis added a project: Cloud-VPS.

Nemo_bis added subscribers: Nemo_bis, Hydriz.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 20 2015, 8:30 AM

This is probably something that appears when the Labs NFS gets a hiccup. Restarting the server should resolve the problem.

Perhaps this is something that can be resolved through puppet runs?

Aklapper added a project: Cloud-Services.Apr 29 2015, 12:18 PM

Is this still happening?

Yes, though the server was restarted in the meanwhile. Deleting the directory is not a problem, if that's the right thing to do. I don't know how to check whether there are others.

$ ls -lh wikistats/
ls: cannot access wikistats/portal: Permission denied
ls: cannot access wikistats/tmp: Permission denied
ls: cannot access wikistats/analytics: Permission denied
ls: cannot access wikistats/lib: Permission denied
ls: cannot access wikistats/reportcard: Permission denied
ls: cannot access wikistats/dumps.old.tmp: Permission denied
ls: cannot access wikistats/dumps: Permission denied
ls: cannot access wikistats/pageviews_reports: Permission denied
ls: cannot access wikistats/animations: Permission denied
ls: cannot access wikistats/dammit.lt: Permission denied
ls: cannot access wikistats/progress: Permission denied
ls: cannot access wikistats/viz_gallery: Permission denied
ls: cannot access wikistats/backup: Permission denied
ls: cannot access wikistats/pediapress: Permission denied
ls: cannot access wikistats/squids: Permission denied
ls: cannot access wikistats/MetricsReporting: Permission denied
ls: cannot access wikistats/_git_sandbox: Permission denied
ls: cannot access wikistats/mail-lists: Permission denied
total 0
?????????? ? ? ? ?            ? analytics
?????????? ? ? ? ?            ? animations
?????????? ? ? ? ?            ? backup
?????????? ? ? ? ?            ? dammit.lt
?????????? ? ? ? ?            ? dumps
?????????? ? ? ? ?            ? dumps.old.tmp
?????????? ? ? ? ?            ? _git_sandbox
?????????? ? ? ? ?            ? lib
?????????? ? ? ? ?            ? mail-lists
?????????? ? ? ? ?            ? MetricsReporting
?????????? ? ? ? ?            ? pageviews_reports
?????????? ? ? ? ?            ? pediapress
?????????? ? ? ? ?            ? portal
?????????? ? ? ? ?            ? progress
?????????? ? ? ? ?            ? reportcard
?????????? ? ? ? ?            ? squids
?????????? ? ? ? ?            ? tmp
?????????? ? ? ? ?            ? viz_gallery
nemobis@dumps-stats:/data/project$ stat wikistats/analytics
stat: cannot stat ‘wikistats/analytics’: Permission denied
nemobis@dumps-stats:/data/project$ sudo stat wikistats/analytics
  File: ‘wikistats/analytics’
  Size: 4096            Blocks: 8          IO Block: 1048576 directory
Device: 1bh/27d Inode: 172032003   Links: 7
Access: (0755/drwxr-xr-x)  Uid: ( 2179/ nemobis)   Gid: (  500/ wikidev)
Access: 2015-06-19 06:58:01.054875389 +0000
Modify: 2013-06-19 06:26:35.000000000 +0000
Change: 2015-06-19 06:58:01.054875389 +0000
 Birth: -
nemobis@dumps-stats:/data/project$ uptime
 12:26:17 up 32 days, 17:26,  1 user,  load average: 1.17, 1.16, 1.24

• chasemp triaged this task as Medium priority.Nov 30 2015, 5:14 PM

• chasemp subscribed.

closing this due to age and activity

The situation is still identical though, so I'd appreciate suggestions on what to do:

nemobis@dumps-stats:/data/project$ ls -lh wikistats/
ls: cannot access wikistats/animations: Permission denied
ls: cannot access wikistats/pageviews_reports: Permission denied
ls: cannot access wikistats/backup: Permission denied
ls: cannot access wikistats/pediapress: Permission denied
ls: cannot access wikistats/squids: Permission denied
ls: cannot access wikistats/portal: Permission denied
ls: cannot access wikistats/dammit.lt: Permission denied
ls: cannot access wikistats/mail-lists: Permission denied
ls: cannot access wikistats/MetricsReporting: Permission denied
ls: cannot access wikistats/_git_sandbox: Permission denied
ls: cannot access wikistats/viz_gallery: Permission denied
ls: cannot access wikistats/lib: Permission denied
ls: cannot access wikistats/analytics: Permission denied
ls: cannot access wikistats/dumps: Permission denied
ls: cannot access wikistats/tmp: Permission denied
ls: cannot access wikistats/progress: Permission denied
ls: cannot access wikistats/reportcard: Permission denied
ls: cannot access wikistats/dumps.old.tmp: Permission denied
total 0
?????????? ? ? ? ?            ? MetricsReporting
?????????? ? ? ? ?            ? _git_sandbox
?????????? ? ? ? ?            ? analytics
?????????? ? ? ? ?            ? animations
?????????? ? ? ? ?            ? backup
?????????? ? ? ? ?            ? dammit.lt
?????????? ? ? ? ?            ? dumps
?????????? ? ? ? ?            ? dumps.old.tmp
?????????? ? ? ? ?            ? lib
?????????? ? ? ? ?            ? mail-lists
?????????? ? ? ? ?            ? pageviews_reports
?????????? ? ? ? ?            ? pediapress
?????????? ? ? ? ?            ? portal
?????????? ? ? ? ?            ? progress
?????????? ? ? ? ?            ? reportcard
?????????? ? ? ? ?            ? squids
?????????? ? ? ? ?            ? tmp
?????????? ? ? ? ?            ? viz_gallery

@Nemo_bis have you stat wikistats/? This is similar to the issue where you don't have the +x permission on the directory:

$ cd /tmp
$ mkdir test
$ touch test/{a,b,c}
$ ls -l test
total 0
-rw-rw-r-- 1 zhuyifei1999 zhuyifei1999 0 Mar 20 22:31 a
-rw-rw-r-- 1 zhuyifei1999 zhuyifei1999 0 Mar 20 22:31 b
-rw-rw-r-- 1 zhuyifei1999 zhuyifei1999 0 Mar 20 22:31 c
$ chmod 644 test
$ ls -l test
ls: cannot access 'test/c': Permission denied
ls: cannot access 'test/b': Permission denied
ls: cannot access 'test/a': Permission denied
total 0
-????????? ? ? ? ?            ? a
-????????? ? ? ? ?            ? b
-????????? ? ? ? ?            ? c

thanks @zhuyifei1999

ignores permissions and does not reproduce root@dumps-stats:/data/project# ls -lh wikistats

reproduces root@dumps-stats:/data/project# sudo -u nemobis ls -lh wikistats

root@dumps-stats:/data/project# chmod +x wikistats/*

now this looks fine:

root@dumps-stats:/data/project# sudo -u nemobis ls -lh wikistats

you'll want to cleanup +x permissions there but this isn't an NFS issue for sure.

Ok. From the description I thought I had checked permissions...

• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:55 PM

Weird state of /data/project for dumps (semi-missing files)Closed, InvalidPublicActions

Description

Event Timeline

Weird state of /data/project for dumps (semi-missing files)
Closed, InvalidPublic
Actions