[Spike] Spark job for digests-only mediawiki-history-reduced
Open, HighPublic
Actions

Description

This task is about moving the mediawiki_history_reduced dataset from using raw-events and digests-events to digests-events only.
Optimizations are needed for the dataset not to grow exponentially with dimension-denormalization, and for the computation not to generate the overly-big intermediate dataset.

Related Objects
Search...

Status	Assigned	Task
Resolved	None	T120037 Vital Signs: Please provide an "all languages" de-duplicated stream for the Community/Content groups of metrics
Resolved	None	T120036 Vital Signs: Please make the data for enwiki and other big wikis less sad, and not just be missing for most days
Open	None	T130256 Wikistats 2.0.
Open	None	T186120 Wikistats Beta
Open	None	T186121 Wikistats 2.0 Remaining reports.
Resolved	None	T188550 Wikistats 2.0: allow to view stats for all language versions (a.k.a. Project families)
Open	None	T188265 Active Editors metric per project family
Resolved	fdans	T189882 Create reports in wikistats UI for "most prolific editors" (a.k.a "top contributors")
Resolved	JAllemandou	T201617 Add AQS endpoint providing top editors ("most prolific contributors") and top pages (by number of edits, by net-bytes-diff and abs-bytes diff)
Open	None	T204965 Create report for "articles with most contributors" in Wikistats2
Open	JAllemandou	T212928 [Spike] Spark job for digests-only mediawiki-history-reduced
Resolved	Amitjoki	T186335 Wikistat Beta: expand topic explorer by default
Duplicate	Erik_Zachte	T191555 stats.wikimedia.org home page should link to wikistats 2

Event Timeline

JAllemandou created this task.Jan 4 2019, 9:56 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 4 2019, 9:56 AM

JAllemandou claimed this task.Jan 4 2019, 9:56 AM

JAllemandou added parent tasks: T204965: Create report for "articles with most contributors" in Wikistats2, T188265: Active Editors metric per project family .

JAllemandou moved this task from Next Up to In Progress on the Analytics-Kanban board.

Milimetric triaged this task as High priority.Jan 7 2019, 4:35 PM

Milimetric moved this task from Incoming to Smart Tools for Better Data on the Analytics board.

JAllemandou moved this task from In Progress to Paused on the Analytics-Kanban board.Feb 7 2019, 8:46 PM

[Spike] Spark job for digests-only mediawiki-history-reduced Open, HighPublicActions

Description

Related ObjectsSearch...

Event Timeline

[Spike] Spark job for digests-only mediawiki-history-reduced
Open, HighPublic
Actions

Related Objects
Search...