I want to start drafting an SLO document this week and would like to validate the direction first.

The mediawiki_content_history iceberg table will serve as the public API boundary for the Dumps 2.0 architecture.

At a high level, we define system health by ensuring that data landing in the iceberg table meets (to be agreed upon) the following criteria:

• Availability: minimal missing events (I am avoiding the term "completeness" on purpose)
• Freshness: minimal lag from mediawiki

We may add additional SLIs based on data quality (DQ), but for now, I’d prefer to keep the two frameworks separate.

Dumps 2.0 boundaries

In terms of systems, the Dumps 2.0 team boundary include:

• Spark jobs and airflow DAGs for batch processing and reconciliation
• mediawiki-event-enrichment application (streaming flink app on dse k8s).

Reliability for these components should be covered by the Dumps 2.0 SLO, either as subcomponents or with dedicated SLOs.

Initially, I propose a single SLO that covers:

• Spark and Airflow jobs (with distinct failure domains for DAGs/tasks where failures may degrade SLIs)
• The Flink app (with its own failure domain and impact on SLIs)
• The mediawiki_content_history table

Client-facing SLIs would apply only to the final table.

Dependencies

Dumps 2.0 systems has dependencies on third party systems (outside team boundaries), not all of which have an SLO

• EventBus and EventGate (covered by T345195: Identify indicators to inform an SLO for event emission and intake - needs finalizing)
• Gobblin + Refine (no SLO)
• Airflow platform (no SLO)
• YARN, HDFS (no SLO)

The fact that dependent systems don't have an SLO is not a blocker, but we should model their reliability expectation explicitly.

The fact that dependent systems don't have an SLO is not a blocker, but we should model their reliability expectation explicitly.

Could you expand on how we would do that?

In T358883#10269971, @xcollazo wrote:

The fact that dependent systems don't have an SLO is not a blocker, but we should model their reliability expectation explicitly.

Could you expand on how we would do that?

As a starting point, Wikimedia's SLOs template recommends:
For dependencies without an SLO yet, or dependencies that habitually miss their SLO, assume that they maintain their historical performance, or worsen slightly but not dramatically

And then we'll adjust after collecting data in the observation period. This is approach is relatively common AFAIK in other parts of our stack.

The ops week effort will pay off big times, we have a good 12 month history of observing systems performance and can provide reasonable estimates for the dependencies listed above (cc / @Ahoelzl ).

Shared link to draft google doc on description above.

SLO docs are on wikitech:

• https://wikitech.wikimedia.org/wiki/SLO/Event_Platform
• https://wikitech.wikimedia.org/wiki/SLO/MediaWiki_Content_History