Create snapshots-validation dag
Closed, InvalidPublic
Actions

Assigned To

None

Authored By

	prabhat
	Aug 12 2025, 10:50 PM

Project Tags

Referenced Files

None

Subscribers

Description

This will cover 2 validations at the moment:

To do

Dag that runs everyday at 12 UTC. Allow configurable airflow variables for:
- Tolerance (start with 10 %)
- Validated projects (default: consider using few main ones enwiki, frwiki, dewiki, etc.)
- Validated namespaces (default [0])
Create tasks to call IsCorrupt and ValidateArticleCounts as per the env variables.
Test dag in dev (using your feature branch)
Tune task timeouts
Deploy snapshotsvalidations service to prod
Deploy dag to prod (this means merging feature -> main)
Test dag in prod

Acceptance criteria

Status	Assigned	Task
Resolved	None	T378598 Business as ususal: Snapshot DAG stability
Invalid	None	T396580 Business as Usual: [Epic] Snapshot improvements and validations [sent to backlog]
Invalid	None	T401751 Create a gRPC endpoint to validate snapshots against corruption
Invalid	None	T401758 Create a gRPC endpoint for snapshot article count validation
Invalid	None	T401761 Create snapshots-validation dag