Page MenuHomePhabricator

Stand up an end-to-end MPIC testing environment in Toolforge
Open, HighPublic

Description

Background

Whilst trying to deploy the MetricsPlatform MediaWiki we discovered that the Beta Cluster cannot make requests to services on the internet. After a couple of conversations with DPE SRE (summarised in T366460#9902517), Data Products have decided to stand up an end-to-end MPIC testing environment in Toolforge.

AC

  • There is a publicly-available test MediaWiki instance
  • The MediaWiki instance submits events to an EventGate instance running in the same cluster
  • The MediaWiki instance requests stream configs from an MPIC instance running in the same cluster

Requirements

  • Documentation
  • QA passed?

Event Timeline

phuedx triaged this task as High priority.

The MediaWiki instance is up and running, pulling instrument configs from the staging MPIC instance running on dse-k8s and generating stream configs.

You can access the the MediaWiki instance here: https://metrics-platform-demo.toolforge.org/wiki/Main_Page
You can verify that stream configs are being generated from instrument configs here: https://metrics-platform-demo.toolforge.org/w/load.php?lang=en&modules=ext.eventLogging&debug=true (search for "streamConfigs":)
You can administer the MediaWiki instance here: https://toolsadmin.wikimedia.org/tools/id/metrics-platform-demo

An important note:

I had to patch EventLogging (see below) to remove its stream config allowlisting behaviour.

EventLogging sends stream configs to the browser so that the JS Event Platform Client can submit events to the event intake service. There are a lot of stream configs that aren't for analytics events so they should be filtered out for performance reasons. However, EventLogging uses an allowlist rather than a denylist, i.e. a stream config isn't sent to the browser by default.

This means that any stream configs generated from the instrument configs fetched from MPIC aren't sent to the browser by default. If we continue dynamically generating stream configs, we will have to update EventLogging accordingly. For now, I've patched EventLogging as follows:

T367860.patch
diff --git a/ServiceWiring.php b/ServiceWiring.php
index 7c7720b..465b959 100644
--- a/ServiceWiring.php
+++ b/ServiceWiring.php
@@ -19,17 +19,9 @@ return [
 			return false;
 		}
 
-		$eventLoggingStreamNames = $services->getMainConfig()
-			->get( 'EventLoggingStreamNames' );
-
-		if ( $eventLoggingStreamNames === false ) {
-			return false;
-		}
-
 		/** @var \MediaWiki\Extension\EventStreamConfig\StreamConfigs $streamConfigs */
-		$streamConfigs = $services->getService( 'EventStreamConfig.StreamConfigs' );
-
-		return $streamConfigs->get( $eventLoggingStreamNames );
+		return $services->getService( 'EventStreamConfig.StreamConfigs' )
+			->get();
 	},
 	'EventLogging.Logger' => static function (): LoggerInterface {
 		return LoggerFactory::getInstance( 'EventLogging' );
  • The MediaWiki instance submits events to an EventGate instance running in the same cluster
  • The MediaWiki instance requests stream configs from an MPIC instance running in the same cluster

This provided difficult to impossible. I discussed this briefly with @WDoranWMF and we decided to:

  • Configure EventLogging to submit events to the event ingestion service running on the Beta Cluster; and
  • Configure MetricsPlatform to fetch instrument configs from the staging MPIC instance running on dse-k8s

This provided difficult to impossible. I discussed this briefly with @WDoranWMF and we decided to:

  • Configure EventLogging to submit events to the event ingestion service running on the Beta Cluster; and

As noted elsewhere, it's important to keep in mind that events submitted to the event ingestion service running on the Beta Cluster will be dropped silently if they are submitted to a stream that is not defined on the metawiki running on the Beta Cluster.

At the moment, we will only be able to say "events are correctly sent to the event ingestion service running on the Beta Cluster" during end-to-end testing.