In T373406#10154077, @Sgs wrote:

Is the code from 1071801 directly accessible in beta and production clusters or do we need to enable there as well?

☝️ Being bold. There are a few cosmetic changes that should make the patch easier to test that I can make before merging it…

@kostajh: I hope you don't mind but I've bold and update the codesearches in the description, scoping them to the EventLogging extension.

In T355837#10147260, @Ottomata wrote:

Oh, it's POST? With a POST body? or is the data just urlencoded query params?

I've tried to capture our discussion in https://wikitech.wikimedia.org/w/index.php?title=Metrics_Platform%2FSampling&wvprov=sticky-header#Experiment_Enrolment_Sampling (the page was moved from Sampling Units). Please review the section and be bold if you spot a mistake.

In T365815#10133670, @phuedx wrote:

Being bold.

Guardrail metrics can be represented by a schema that can be ingested by GrowthBook.

In T374471#10140440, @Urbanecm_WMF wrote:

• Development and QA: Currently, people are able to run ge.utils.setUserVariant('control') in console and switch between buckets as they please. This means we can avoid creating dozens of accounts, hoping you get into the "right" bucket (or worse, trying to change the user ID).
• Outreach events: In the past, we've been asked by affiliates to make it possible to provide single experience for all attendees of their event. We fulfilled that need with geForceVariant, which is a GET parameter for Special:CreateAccount. This need is especially important if the organisers actively talk about the A/B tested feature at the event (it is nearly impossible to do if half attendees do not have the feature available).

The integration test phase is over. We no longer need to do this.

The integration test phase is over. We no longer need to do this.

The integration test phase is over. We no longer need to do this.

The integration test phase is over. We no longer need to do this.

I think this is blocked on T371498: Metrics Platform Integration: PoC for tracking impressions of the Community Updates homepage module, specifically merging https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventLogging/+/1070992.

In T374464#10137578, @mforns wrote:

I've looked for other instances that we've had to generate fake event data from a jsonschema and could not find any examples...

@Sfaci: I took a look. Thanks for detailing the options that we have. Do you have any preference? Mine would be to stay close to what other teams are doing with similar systems but also to try to keep it as simple as possible.

In T374251#10132289, @nettrom_WMF wrote:

I'm not sure we need to? MP has performer.pageview_id (1.2.0.yaml L446) It seems we might need a particular stream configuration for it to be present, though. That's something for us to verify with the folks in Data Products, so I'm pinging @phuedx and @mforns to get it on their radar.

Does this task cover generating fake data?

Guardrail metrics can be represented by a schema that can be ingested by GrowthBook.

Does one stream correspond to one instrument?

In T370183#10131911, @VirginiaPoundstone wrote:

Having the data dictionary would be a really useful point of entry and reference.

I'm holding off on this until next week's sprint with Growth.

@VirginiaPoundstone: As discussed, I've added a patch containing the sample CTR instrument ready for review/refinement during the sprint with Growth.

This task is blocked on the review of https://gerrit.wikimedia.org/r/c/schemas/event/secondary/+/1067306 for T368326: Update Metrics Platform Client Libraries to accept experiment membership. Ideally, we can merge all schema updates at once so as to avoid having to update the schema IDs embedded in the client libraries multiple times.

@phuedx What are best steps to advance this SPIKE?

In T365473#10084886, @cjming wrote:

Renaming contextual_attributes and propagating that change throughout all the schemas and client libraries and mpic and current instruments in prod feels like adding to an already full plate atm which is also why I'm fine with leaving it as is and giving it a clear definition in docs. But maybe there's never a good time to do a name change like this and better now than later when we have dozens more instruments in prod.

Thanks, @Sfaci. I'll rebase and finish the TODOs on !39

Moving to Code Review / Tech Input for code review of MR 42.

While I originally settled on contextual attributes, my preferences are: 1) context fields; and 2) context values. I have a slight preference for 1 because it matches the verbiage when talking about events and event schemas (events have fields, event schemas have fields).

Are they still called contextual attributes in the current version of Metrics Platform?

• plus performance, and scalability evaluation to the best of our ability.

MR 39 is blocked on MR 42, which is now ready for review (thanks for the ping @Sfaci!). I'll update the task description…

So the only way that we ensure consistency of assignment output sans backend store is by locking all the inputs.

In T372368#10069353, @Aklapper wrote:

See https://www.mediawiki.org/wiki/Phabricator/Project_management#Types_of_Projects for our taxonomy.

• Can we successfully bucket users based on traffic?

In T372368#10061580, @Jdforrester-WMF wrote:

In T372368#10060455, @Aklapper wrote:

https://phabricator.wikimedia.org/project/profile/5324/ already exists, so I'd rather call this #mediawiki-extensions-MetricsPlatform to avoid confusion?

I think maybe that project is wrongly-configured? It appears to be a team backlog, but it's got the icon and colour of a component.

Usecases examined

After a little more thought, we can combine the initial approach (fetching and caching the config in a per-DC cache on a loop) with the above approach to ensure that we don't contact Statsig whilst serving the page.

@cjming: I've added a bunch of comments and taken the liberty of updating the example CTR instrument. Would it be helpful to pull out all of the open questions into a separate section so that they can be addressed (or converted into Phab tasks, emails, etc)?

I think that this could be moved to done and any further deployment/responding to points in the Application Security Review can be done in other tasks /cc @VirginiaPoundstone @WDoranWMF