Maniphest T192474

Migrate pagecounts-ez generation to hadoop
Open, HighPublic
Actions

Assigned To

None

Authored By

	• Nuria
	Apr 18 2018, 5:29 PM

Details

	Subject	Repo	Branch	Lines +/-
	[WIP] Migrate pagecounts-ez to hadoop	operations/puppet	production	+1 -1
	Encode to pagecounts-ez format with a UDF	analytics/refinery/source	master	+259 -1

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T251777 Creation of canonical pageview dumps for users to download
Open	None	T192474 Migrate pagecounts-ez generation to hadoop
Open	None	T188041 Generate pagecounts-ez data back to 2008
Resolved	• fdans	T249984 Document missing project types in pagecount dumps
Resolved	• fdans	T252857 Create job that backfills Pagecounts-EZ (2011 - 2016) data via hadoop correcting issues
Resolved	JAllemandou	T265732 Create monthly job for canonical pageviews
Open	None	T283593 Add ignore success flags option to pageview monthly dumps

Event Timeline

• Nuria created this task.Apr 18 2018, 5:29 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 18 2018, 5:29 PM

leila added a project: Research.Apr 18 2018, 5:37 PM

leila subscribed.

• fdans triaged this task as Low priority.Apr 19 2018, 4:43 PM

• fdans moved this task from Incoming to Operational Excellence on the Analytics board.

Milimetric added a subtask: T188041: Generate pagecounts-ez data back to 2008.Apr 23 2018, 4:59 PM

• Nuria raised the priority of this task from Low to High.Sep 26 2018, 7:17 PM

• Nuria moved this task from Operational Excellence to Data Quality on the Analytics board.

• Nuria moved this task from Data Quality to Smart Tools for Better Data on the Analytics board.

Milimetric renamed this task from Refactor pagecounts-ez generation to Migrate pagecounts-ez generation to hadoop.Oct 18 2018, 5:38 PM

Milimetric lowered the priority of this task from High to Medium.

@Nuria what shall we do with this task? Anything Research can help with?

We have no plans to tackle this for the next couple of quarters.

@Nuria Ok. I'll decline then and feel free to re-open if things change.

leila closed this task as Declined.Apr 2 2019, 12:05 AM

Milimetric mentioned this in T238243: Archive /home/ezachte data on stat1007.Jan 3 2020, 4:41 PM

• Nuria reopened this task as Open.Jan 4 2020, 9:32 AM

Milimetric claimed this task.Jan 6 2020, 1:54 AM

Milimetric raised the priority of this task from Medium to High.

Milimetric added a project: Analytics-Kanban.

Milimetric moved this task from Next Up to In Progress on the Analytics-Kanban board.

Change 562079 had a related patch set uploaded (by Milimetric; owner: Milimetric):
[operations/puppet@production] [WIP] Migrate pagecounts-ez to hadoop

https://gerrit.wikimedia.org/r/562079

gerritbot added a project: Patch-For-Review.Jan 6 2020, 1:55 AM

Change 562597 had a related patch set uploaded (by Milimetric; owner: Milimetric):
[analytics/refinery/source@master] Encode to pagecounts-ez format with a UDF

https://gerrit.wikimedia.org/r/562597

Milimetric moved this task from In Progress to Paused on the Analytics-Kanban board.Feb 28 2020, 3:12 PM

Milimetric moved this task from Paused to In Progress on the Analytics-Kanban board.Feb 28 2020, 8:25 PM

• Nuria moved this task from In Progress to Paused on the Analytics-Kanban board.Mar 12 2020, 4:07 PM

Milimetric added a subtask: T249984: Document missing project types in pagecount dumps.Apr 13 2020, 3:23 PM

leila removed a project: Research.Apr 13 2020, 9:20 PM

• fdans added a parent task: T251777: Creation of canonical pageview dumps for users to download.May 5 2020, 8:42 AM

Is this change from @Milimetric now abandoned? https://gerrit.wikimedia.org/r/562079

Change 562079 abandoned by Milimetric:
[WIP] Migrate pagecounts-ez to hadoop

Reason:
This change is still relevant if someone needs to find the correct archive message. But I'm abandoning because we're deprecating this and similar datasets and publishing a more comprehensive one instead.

https://gerrit.wikimedia.org/r/562079

Milimetric reassigned this task from Milimetric to • fdans.Jun 8 2020, 4:38 PM

Milimetric moved this task from Paused to Parent Tasks on the Analytics-Kanban board.

Milimetric subscribed.

• Nuria closed subtask T252857: Create job that backfills Pagecounts-EZ (2011 - 2016) data via hadoop correcting issues as Resolved.Jul 6 2020, 10:11 PM

Mascia.mauro subscribed.Sep 30 2020, 10:29 AM

Mascia.mauro mentioned this in T271521: Hourly pageviews stop working after the last pageviews-20210106-140000.gz.Jan 8 2021, 10:32 AM

• fdans closed subtask T249984: Document missing project types in pagecount dumps as Resolved.Jan 25 2021, 7:01 PM