Page MenuHomePhabricator

Put pageviews dataset in labs /public/dumps
Closed, ResolvedPublic2 Story Points

Description

Pagecounts dataset has been deprecated, but it used to be automatically synced to labs at /public/dumps. We should make sure they same happens for the new pageviews dataset.

This should be changed:
https://github.com/wikimedia/operations-puppet/blob/f4f87efce1fcaab1296f2b9ee31a429b391af6c3/modules/role/manifests/labs/nfs/misc.pp#L25

Event Timeline

Stigmj created this task.Aug 11 2016, 1:23 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 11 2016, 1:23 AM

See this announcement on the Analytics public mailing list:

https://lists.wikimedia.org/pipermail/analytics/2016-August/005339.html

Could we perhaps get https://dumps.wikimedia.org/other/pageviews into /public/dumps/pageviews then?

Ottomata renamed this task from pagecount-dumps at /public/dumps/pagecounts-* has stopped to Put pageviews dataset in labs /public/dumps.Aug 15 2016, 3:22 PM
Ottomata updated the task description. (Show Details)
-jem- added a subscriber: -jem-.Aug 16 2016, 1:03 PM

Any chance for this to be expedited? At nowiki we are currently using these datasets for a local stats-service (https://tools.wmflabs.org/pagecount/) and the users are complaining (https://no.wikipedia.org/wiki/Wikipedia:Torget#Sidevisningsstatistikk)...

Change 307502 had a related patch set uploaded (by Ottomata):
Rsync pageviews to labs nfs hosts

https://gerrit.wikimedia.org/r/307502

Ottomata triaged this task as High priority.
Ottomata edited projects, added Analytics-Kanban; removed Analytics.
Ottomata set the point value for this task to 2.

Change 307502 merged by Ottomata:
Rsync pageviews to labs nfs hosts

https://gerrit.wikimedia.org/r/307502

Ottomata moved this task from Next Up to Done on the Analytics-Kanban board.Aug 30 2016, 12:59 PM

Done! I'm running the first rsync over now.

I didn't create a specific rsync module, it seems like the ::dumps one should be enough.

@Stigmj, I don't have a full understanding of how the /public/dumps NFS mount works in labs, but I think that /public/dumps/pageviews should exist now. Can you confirm?

Nuria closed this task as Resolved.Aug 30 2016, 4:09 PM

@Ottomata yeah they are there. Thank you.