Clickstream datasets only reference 'other' link type, no 'link'
Open, Needs TriagePublic
Actions

Assigned To

Authored By

	JAllemandou
	May 28 2024, 7:43 AM

Description

The clickstream datasets generated for month 2024-04 have a number of lines that matches previous months, but the between-pages links referenced are all flagged other while they should be flagged link.
This is a data-dependency issue: the job is not waiting for the linktarget table to be present in Hive before starting.

Details

	Subject	Repo	Branch	Lines +/-
	Update clickstream job - better joins	analytics/refinery/source	master	+23 -9

Customize query in gerrit

	Title	Reference	Author	Source Branch	Dest Branch
	Fix analytics clickstream job missing a sensor	repos/data-engineering/airflow-dags!709	joal	fix_clickstream	main

Customize query in GitLab

Event Timeline

JAllemandou created this task.May 28 2024, 7:43 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 28 2024, 7:43 AM

JAllemandou claimed this task.May 28 2024, 7:45 AM

JAllemandou edited projects, added Data-Engineering (Q4 2024 April 1st - June 30th); removed Data-Engineering.

JAllemandou updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-analytics) [2024-05-28T07:53:01Z] <joal> manually rerun clickstream job for 2024-04 to pick up linktarget data that was not present at the moment it ran automatically (T366042)