As part of the topictrends project(Content Observability hypothesis, WE2.3.13), pageviews data dumps are analysed daily. Currently we are fetching it from https://dumps.wikimedia.org/other/pageview_complete but it would be much better if we can access it from NFS mount (also in the context of recent traffic restrictions to dumps.wikimedia.org)
Project ID: language
Compute Instance Name: language-lab
This project already has mount_nfs: true in the Hiera Config
| Subject | Author | Repo | Branch | Lines +/- | |
|---|---|---|---|---|---|
| cloudnfs: add dumps to language project | Volans | operations/puppet | production | +2 -1 |
Change #1303479 had a related patch set uploaded (by Volans; author: Volans):
[operations/puppet@production] cloudnfs: add dumps to language project
Change #1303479 merged by Volans:
[operations/puppet@production] cloudnfs: add dumps to language project
@santhosh as mount_nfs: true is set at project puppet level, the dumps NFS will be mounted on all instances in the project.
The above patch has been merged, within the next 30~40 minutes it should be applied to the NFS server and a subsequent puppet run on your instance(s) should mount it.
Leaving the task open for now until we can confirm that it's mounted and works fine.
The dumps seems to be mounted and accessible, I'm tentatively resolving this @santhosh , feel free to re-open it if you encounter any issue:
language-lab# df -hT | grep dumps clouddumps1001.wikimedia.org:/ nfs4 196T 147T 39T 80% /mnt/nfs/dumps-clouddumps1001.wikimedia.org clouddumps1002.wikimedia.org:/ nfs4 196T 147T 39T 80% /mnt/nfs/dumps-clouddumps1002.wikimedia.org language-lab# ls -la /mnt/nfs/dumps-clouddumps1001.wikimedia.org total 876616 drwxr-xr-x 1102 400 400 36864 Jun 17 04:01 . drwxr-xr-x 5 root root 4096 Jun 17 16:09 .. drwxrwxr-x 4 400 400 4096 Nov 22 2019 10wikipedia -rw-rw-r-- 1 400 400 127 Jul 17 2009 404.html lrwxrwxrwx 1 root root 40 Aug 6 2013 PlayBook -> /data/xmldatadumps/public/other/PlayBook drwxrwxr-x 13 400 400 4096 Jun 1 21:42 aawiki drwxrwxr-x 13 400 400 4096 Jun 1 21:08 aawikibooks ....