Page MenuHomePhabricator

Configure experiments with stream, schema and contextual attributes
Open, HighPublic8 Estimated Story Points

Description

Description

At this time instruments can override the contextual attributes statically defined in the product_metrics.web.base stream with those ones configured in xLab. That way every instrument can use their own configuration regarding which data will be collected.

The purpose of this ticket is to implement that also for experiments. We are already exploring T407882: Instrumentation Specification for the suite of standard instruments/components where an instrument suite will be defined and created to be used by experiments. That means that, depending on which metrics are needed by an experiment, this one should be configured with a specific set of contextual attributes. So to configure experiments same way as instrument are being configured right now would be very interesting.

The main benefit will be that experiment owners won't need to configure a custom stream when they need a different combination of contextual attributes as the ones configured in product_metrics.web_base by default. The number of custom streams would be reduced considerably.

Acceptance criteria

  • Experiments can be configured using xLab by defining their own stream, schema and contextual attributes
  • The way MetricsPlatform instantiates an experiment has been redefined for experiments to be able to use their own configuration (stream, schema, contextual attributes) when sending events
  • All the validation rules related to the Regulation Section, Data Collection Guidelines and the combination of contextual attributes apply now also for experiments

Event Timeline

Does The Vision™, as currently stated, obviate the need for this @Sfaci ?

Does The Vision™, as currently stated, obviate the need for this @Sfaci ?

I'm sorry for answering a bit late. I didn't realize!

I would say that we still need this, even considering The Vision™. We are talking here about experiments being able to use their own combination of contextual attributes as instruments can do thanks to T398480: [Epic] Update instruments configuration for xLab where we defined that a stream could have more than one instrument. This task aims to enable experiments to do that. And this also would prevent teams from creating custom streams only to be able to use a different combination of contextual attributes than the one that base schema provides.

Regarding the xLab UI part, it would die pretty soon because of GrowthBook but the relevant changes in MetricsPlatform and client libraries would survive

JVanderhoop-WMF moved this task from Incoming to READY TO GROOM on the Test Kitchen board.

It seems that there is a lot of short term pain here that is requiring new streams.

Milimetric set the point value for this task to 8.Tue, Dec 2, 4:47 PM
Milimetric moved this task from READY TO GROOM to Backlog on the Test Kitchen board.