i noticed today that the cleanup_xmldumps timer on dumpsdata1005 has been failing for the past two days with he following error
Mar 03 09:25:00 dumpsdata1005 systemd[1]: Starting Regular jobs to clean up xml dumps... Mar 03 09:25:00 dumpsdata1005 python3[24363]: Traceback (most recent call last): Mar 03 09:25:00 dumpsdata1005 python3[24363]: File "/usr/local/bin/cleanup_old_xmldumps.py", line 465, in <module> Mar 03 09:25:00 dumpsdata1005 python3[24363]: main() Mar 03 09:25:00 dumpsdata1005 python3[24363]: File "/usr/local/bin/cleanup_old_xmldumps.py", line 461, in main Mar 03 09:25:00 dumpsdata1005 python3[24363]: cleaner.clean() Mar 03 09:25:00 dumpsdata1005 python3[24363]: File "/usr/local/bin/cleanup_old_xmldumps.py", line 402, in clean Mar 03 09:25:00 dumpsdata1005 python3[24363]: self.cleanup_wiki(tokeep, partials, wiki) Mar 03 09:25:00 dumpsdata1005 python3[24363]: File "/usr/local/bin/cleanup_old_xmldumps.py", line 367, in cleanup_wiki Mar 03 09:25:00 dumpsdata1005 python3[24363]: dumps = self.get_dumps(wiki) Mar 03 09:25:00 dumpsdata1005 python3[24363]: File "/usr/local/bin/cleanup_old_xmldumps.py", line 230, in get_dumps Mar 03 09:25:00 dumpsdata1005 python3[24363]: dirs = os.listdir(path) Mar 03 09:25:00 dumpsdata1005 python3[24363]: PermissionError: [Errno 13] Permission denied: '/data/xmldatadumps/public/zawikibooks' Mar 03 09:25:00 dumpsdata1005 systemd[1]: cleanup_xmldumps.service: Main process exited, code=exited, status=1/FAILURE Mar 03 09:25:00 dumpsdata1005 systemd[1]: cleanup_xmldumps.service: Failed with result 'exit-code'. Mar 03 09:25:00 dumpsdata1005 systemd[1]: Failed to start Regular jobs to clean up xml dumps.
Looking in /data/xmldatadumps/public/ i notice that all folderes lexagraphocaly larger then wikimania2005 wiki are only readable by root
$ ls -la /data/xmldatadumps/public total 33468 drwxr-xr-x 1031 dumpsgen dumpsgen 32768 Mar 1 07:00 . drwxr-xr-x 5 dumpsgen dumpsgen 4096 Mar 2 07:37 .. drwxrwxr-x 2 dumpsgen dumpsgen 4096 Nov 22 2019 10wikipedia -rw-rw-r-- 1 dumpsgen dumpsgen 127 Jul 17 2009 404.html drwxrwxr-x 7 dumpsgen dumpsgen 4096 Feb 20 09:44 aawiki ..... SNIP ..... drwxrwxr-x 7 dumpsgen dumpsgen 4096 Feb 20 09:30 wbwikimedia drwxrwxr-x 7 dumpsgen dumpsgen 4096 Feb 20 09:35 wikidatawiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2005wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2006wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2007wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2008wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2009wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2010wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2011wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2012wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2013wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2014wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2015wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2016wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2017wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimania2018wiki drwx------ 2 root root 4096 Mar 2 08:15 wikimaniawiki drwx------ 2 root root 4096 Mar 2 08:15 wowiki drwx------ 2 root root 4096 Mar 2 08:15 wowikiquote drwx------ 2 root root 4096 Mar 2 08:15 wowiktionary drwx------ 2 root root 4096 Mar 2 08:15 wuuwiki drwx------ 2 root root 4096 Mar 2 08:15 wuuwiktionary drwx------ 2 root root 4096 Mar 2 08:15 xalwiki drwx------ 2 root root 4096 Mar 2 08:15 xalwiktionary drwx------ 2 root root 4096 Mar 2 08:15 xhwiki drwx------ 2 root root 4096 Mar 2 08:15 xhwikibooks drwx------ 2 root root 4096 Mar 2 08:15 xhwiktionary drwx------ 2 root root 4096 Mar 2 08:15 xmfwiki drwx------ 2 root root 4096 Mar 2 08:15 yiwiki drwx------ 2 root root 4096 Mar 2 08:15 yiwikisource drwx------ 2 root root 4096 Mar 2 08:15 yiwiktionary drwx------ 2 root root 4096 Mar 2 08:15 yowiki drwx------ 2 root root 4096 Mar 2 08:15 yowikibooks drwx------ 2 root root 4096 Mar 2 08:15 yowiktionary drwx------ 2 root root 4096 Mar 2 08:15 yuewiktionary drwx------ 2 root root 4096 Mar 2 08:15 zawiki drwx------ 2 root root 4096 Mar 2 08:15 zawikibooks drwx------ 2 root root 4096 Mar 2 08:15 zawikiquote drwx------ 2 root root 4096 Mar 2 08:15 zawiktionary drwx------ 2 root root 4096 Mar 2 08:15 zeawiki drwx------ 2 root root 4096 Mar 2 08:15 zeawiktionary drwx------ 2 root root 4096 Mar 2 08:15 zh_classicalwiki drwx------ 2 root root 4096 Mar 2 08:15 zh_classicalwiktionary drwx------ 2 root root 4096 Mar 2 08:15 zh_min-nanwiki drwx------ 2 root root 4096 Mar 2 08:15 zh_min_nanwiki drwx------ 2 root root 4096 Mar 2 08:15 zh_min_nanwikibooks drwx------ 2 root root 4096 Mar 2 08:15 zh_min_nanwikiquote drwx------ 2 root root 4096 Mar 2 08:15 zh_min_nanwikisource drwx------ 2 root root 4096 Mar 2 08:15 zh_min-nanwiktionary drwx------ 2 root root 4096 Mar 2 08:15 zh_min_nanwiktionary drwx------ 2 root root 4096 Mar 2 08:15 zhwiki drwx------ 2 root root 4096 Mar 2 08:15 zhwikibooks drwx------ 2 root root 4096 Mar 2 08:15 zhwikinews drwx------ 2 root root 4096 Mar 2 08:15 zhwikiquote drwx------ 2 root root 4096 Mar 2 08:15 zhwikisource drwx------ 2 root root 4096 Mar 2 08:15 zhwikiversity drwx------ 2 root root 4096 Mar 2 08:15 zhwikivoyage drwx------ 2 root root 4096 Mar 2 08:15 zhwiktionary drwx------ 2 root root 4096 Mar 2 08:15 zh_yuewiki drwx------ 2 root root 4096 Mar 2 08:15 zh_yuewiktionary drwx------ 2 root root 4096 Mar 2 08:15 zuwiki drwx------ 2 root root 4096 Mar 2 08:15 zuwikibooks drwx------ 2 root root 4096 Mar 2 08:15 zuwiktionary