Maniphest T341725

WikiLambda metrics: Add wikilambda_zobject_labels query to sqoop
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	DMartin-WMF
	Jul 12 2023, 8:25 PM

Tags

Referenced Files

None

Subscribers

Description

Description

Add a query to sqoop.py (analytics/refinery repo) for periodic extraction of data from the table wikilambda_zobject_labels, to be sent to the data lake. An example query is given by this patch.

The table is declared here.

Consider whether to extract everything, or possibly omit some columns from the extraction.

Completion checklist

Before closing this task, review one by one the checklist available here: https://www.mediawiki.org/wiki/Abstract_Wikipedia_team/Definition_of_Done#Front-end_Task/Bug_Completion_Checklist

Details

	Subject	Repo	Branch	Lines +/-
	Add wikilambda_zobject_labels & wikilambda_zobject_function_join queries to sqoop	analytics/refinery	master	+46 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open	BUG REPORT	None	T344206 Special:MathStatus exception error on Wikifunctions
		Resolved		Jdforrester-WMF	T342865 Post-creation work for wikifunctionswiki
		Resolved		Jdforrester-WMF	T323399 Add Wikifunctions as a sibling project to the Portals once it launches
		Resolved		cmassaro	T342820 Migrate wikifunctions.org from locked-down to limited mode, letting users edit wikitext pages and some
		Resolved		DMartin-WMF	T341748 Wikifunctions metrics: Track inventory of functions, implementations, tests
		Resolved		DMartin-WMF	T341725 WikiLambda metrics: Add wikilambda_zobject_labels query to sqoop

Event Timeline

DMartin-WMF created this task.Jul 12 2023, 8:25 PM

DMartin-WMF mentioned this in T341729: WikiLambda metrics: Create wikilambda_zobject_labels table in Hive.Jul 12 2023, 8:40 PM

DMartin-WMF added a parent task: T341748: Wikifunctions metrics: Track inventory of functions, implementations, tests.Jul 12 2023, 11:15 PM

Change 937568 had a related patch set uploaded (by David Martin; author: David Martin):

[analytics/refinery@master] Add wikilambda_zobject_labels query to sqoop

https://gerrit.wikimedia.org/r/937568

gerritbot added a project: Patch-For-Review.Jul 13 2023, 3:25 AM

Jdforrester-WMF changed the task status from Open to In Progress.Jul 13 2023, 4:49 PM

Jdforrester-WMF moved this task from To triage to Phase λ – Launch on the Abstract Wikipedia team board.

Jdforrester-WMF edited projects, added Abstract Wikipedia team (Phase λ – Launch); removed Abstract Wikipedia team.

Jdforrester-WMF moved this task from Incoming to In progress on the Abstract Wikipedia team (Phase λ – Launch) board.

DMartin-WMF added a project: Data Products.Jul 14 2023, 6:14 PM

DMartin-WMF claimed this task.Jul 17 2023, 8:54 PM

Change 937568 merged by Milimetric:

[analytics/refinery@master] Add wikilambda_zobject_labels & wikilambda_zobject_function_join queries to sqoop

https://gerrit.wikimedia.org/r/937568

Maintenance_bot removed a project: Patch-For-Review.Jul 18 2023, 6:11 PM

Jdforrester-WMF moved this task from In progress to QA needed on the Abstract Wikipedia team (Phase λ – Launch) board.Jul 19 2023, 12:01 AM

DMartin-WMF mentioned this in T342199: WikiLambda metrics: Create puppet scripting for Wikifunctions tables.Jul 19 2023, 2:27 AM

DVrandecic closed this task as Resolved.Jul 19 2023, 4:01 AM

DMartin-WMF mentioned this in T355738: WikiLambda metrics: Track localization of types in Wikifunctions.Jan 24 2024, 3:14 AM

KCVelaga_WMF mentioned this in T365813: Develop a unified Content Translation (CX) monitoring dashboard.May 24 2024, 1:17 PM