[Epic] Create Lifecycle Diagrams for Webrequests
Open, Needs TriagePublic
Actions

Assigned To

None

Authored By

	VirginiaPoundstone
	Tue, Apr 30, 2:59 PM

Description

Background/Goal

In order to transfer knowledge about Webrequests and Pageviews, we need diagrams that document the data lifecycle and infrastructure

KR/Hypothesis(Initiative)

Not yet prioritized within OKR level work. This falls into Essential Work.

Success metrics

How we will measure success

Example areas:

Timeboxed to 4 hours
Presentation given and recorded for onboarding purposes
DPE Team is satisfied and demonstrates understanding of the lifecycle
Diagrams have a maintenance plan

In scope

Lifecycle for all 21 technologies and 36 datasets that are part of webrequests is included

Out of Scope

anything outside webrequests

Artifacts & Resources

See examples: https://wikitech.wikimedia.org/wiki/Data_Engineering/Systems/Hadoop_Event_Ingestion_Lifecycle

pipeline tree: https://miro.com/app/board/uXjVNG_RlDk=/

Webrequest intro slides: https://docs.google.com/presentation/d/1UKISVD5FEhBNDrxX-_sBYY8Xno0AR-J7fU32k_c-ung/edit#slide=id.g2d09cd1c67a_0_71

Event Timeline

VirginiaPoundstone created this task.Tue, Apr 30, 2:59 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptTue, Apr 30, 2:59 PM

VirginiaPoundstone moved this task from Incoming to Pipelines Backlog on the Data Products board.Tue, Apr 30, 2:59 PM

FYI - as part of some of the data quality work for webrequests the team updated this page with more details:

https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Webrequest

[Epic] Create Lifecycle Diagrams for Webrequests Open, Needs TriagePublicActions