I did some experiments. Seems that I can publish new files but cannot overwrite the existing files. Also manual sync-up didn't reflect immediately. I am wondering how often the auto sync up is at stat1005.

Thanks,
Jennifer

The /srv/published sync thing is complicated in that it is multi-source, and if there are files with the same path names, which one gets synced is not defined (but I'd guess that the latest alphabetically sorted hostname would win). In this case, /srv/published/notebooks/weekly_edits/weekly_edits_summary.html exists on both stat1005 and stat1007, and it looks like the stat1007 version wins (owned there by @SNowick_WMF).

You should delete whichever one you don't want.

I inadvertently copied extra files from notebook1004 to that dir, I have deleted the weekly_edits directory on stat1007.

Thank you @Ottomata, @SNowick_WMF

How often do we auto sync now?

Perhaps related but @SNowick_WMF and I ran into a perplexing permissions issue. When I tried to run published-sync on stat1007 I got the following:

/usr/bin/flock -n /var/lock/published-sync -c /usr/bin/rsync -rptL -v --delete /srv/published/ thorium.eqiad.wmnet::published-destination/stat1007//
sending incremental file list
rsync: readlink_stat("/srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs/related_pages.csv") failed: Permission denied (13)
rsync: readlink_stat("/srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs/pageview_archive.csv") failed: Permission denied (13)
IO error encountered -- skipping file deletion

and ls -lah /srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs/ has the following output for me:

ls: cannot access '/srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs/.': Permission denied
ls: cannot access '/srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs/..': Permission denied
ls: cannot access '/srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs/related_pages.csv': Permission denied
ls: cannot access '/srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs/pageview_archive.csv': Permission denied
total 0
d????????? ? ? ? ?            ? .
d????????? ? ? ? ?            ? ..
-????????? ? ? ? ?            ? pageview_archive.csv
-????????? ? ? ? ?            ? related_pages.csv

but then when Shay runs it (after fixing the ownership with chown -R :wikidev /srv/published/dashboards/Wikipedia_C-19_Comms_Stats):

drwxrw-r-- 2 snowick wikidev 4.0K May 15 08:14 .
drwxrwxr-x 3 snowick wikidev 4.0K May 28 04:09 ..
-rw-r--r-- 1 snowick wikidev 3.0K May 15 07:02 pageview_archive.csv
-rw-r--r-- 1 snowick wikidev 1.1M May 15 08:14 related_pages.csv

So I should be able to see it since I'm in wikidev too, but for some reason not? And published-sync continues to complain about those files.

How often do we auto sync now?

Every 15 minutes. Your files have been synced, but I think that varnish must just be caching the old ones. Let's check tomorrow.

a perplexing permissions issue

Hm! Weird, the docs directory didn't have group executable perms, which means you can't read contents of the directory.

$ sudo chmod g+x /srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs
$ ls -l /srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs
total 1032
-rw-r--r-- 1 snowick wikidev    2972 May 15 07:02 pageview_archive.csv
-rw-r--r-- 1 snowick wikidev 1052663 May 15 08:14 related_pages.csv

a perplexing permissions issue

Hm! Weird, the docs directory didn't have group executable perms, which means you can't read contents of the directory.

$ sudo chmod g+x /srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs
$ ls -l /srv/published/dashboards/Wikipedia_C-19_Comms_Stats/docs
total 1032
-rw-r--r-- 1 snowick wikidev    2972 May 15 07:02 pageview_archive.csv
-rw-r--r-- 1 snowick wikidev 1052663 May 15 08:14 related_pages.csv

published-sync doesn't have any problems now! Thank you, @Ottomata!