Maniphest T429253

data-reload: implement export_n3 task
Closed, ResolvedPublic1 Estimated Story Points
Actions

Assigned To

Authored By

	gmodena
	Jun 15 2026, 7:56 PM

Project Tags

Referenced Files

None

Subscribers

Description

We need an export_n3 Airflow task runs

org.wikidata.query.rdf.spark.transform.structureddata.dumps.NTripleGenerator

That stores n3 triple files in S3.

References:

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		gmodena	T431253 [DE2.4.2] WDQS v2 Scaling
		Open		gmodena	T440523 [DE2.6.3] Achieve Launch Readiness for GA
		Stalled		gmodena	T422179 Set up regular bulk data ingestion and indexing pipeline
		Resolved		gmodena	T428237 data-reload: spark processing jobs need to be orchestrated in the wikidata dag.
		Resolved		gmodena	T429253 data-reload: implement export_n3 task
		Resolved		gmodena	T429254 data-reload: keep data pipeline tables DDL under version control

Event Timeline

gmodena created this task.Jun 15 2026, 7:56 PM

gmodena created subtask T429254: data-reload: keep data pipeline tables DDL under version control.Jun 15 2026, 8:00 PM

gmodena updated the task description. (Show Details)

gmodena mentioned this in T428237: data-reload: spark processing jobs need to be orchestrated in the wikidata dag..Jun 16 2026, 12:36 PM

gmodena mentioned this in T428235: [NEEDS GROOMING] data-reload: create a QLever index using an ephemeral instance.Jun 16 2026, 12:40 PM

gmodena claimed this task.Jun 17 2026, 11:25 AM

gmodena edited projects, added: Wikidata Platform Team (Sprint 06 (2026/06/02)); removed: Wikidata Platform Team.

gmodena set the point value for this task to 1.

gmodena moved this task from Backlog to In Engineering on the Wikidata Platform Team (Sprint 06 (2026/06/02)) board.

gmodena opened https://gitlab.wikimedia.org/repos/wikidata-platform/wdqs/wdqs-data-processing/-/merge_requests/3

dumps: make --filname-format optional

gmodena updated https://gitlab.wikimedia.org/repos/wikidata-platform/wdqs/wdqs-data-processing/-/merge_requests/4

memory optimzied build

gmodena moved this task from In Engineering to In Code Review on the Wikidata Platform Team (Sprint 06 (2026/06/02)) board.Jun 23 2026, 12:34 PM

gmodena merged https://gitlab.wikimedia.org/repos/wikidata-platform/wdqs/wdqs-data-processing/-/merge_requests/4

memory optimzied build

BTracy-WMF moved this task from Sprint 06 (2026/06/02) to Sprint 07 (2026/07/07 on the Wikidata Platform Team board.Jul 7 2026, 3:15 PM

BTracy-WMF edited projects, added: Wikidata Platform Team (Sprint 07 (2026/07/07); removed: Wikidata Platform Team (Sprint 06 (2026/06/02)).

BTracy-WMF moved this task from Backlog to In Code Review on the Wikidata Platform Team (Sprint 07 (2026/07/07) board.

trueg moved this task from In Code Review to Needs Sign-off on the Wikidata Platform Team (Sprint 07 (2026/07/07) board.Jul 29 2026, 3:27 PM

BTracy-WMF closed this task as Resolved.Aug 3 2026, 8:22 PM

BTracy-WMF closed subtask T429254: data-reload: keep data pipeline tables DDL under version control as Resolved.

gmodena merged https://gitlab.wikimedia.org/repos/wikidata-platform/wdqs/wdqs-data-processing/-/merge_requests/3

dumps: make --filname-format optional