Page MenuHomePhabricator

SDS 2.1.1 Evaluations of 3rd part Experimentation Platform by Traffic
Closed, ResolvedPublic

Description

Description

As part of OKR hypothesis 2.1.1, "If we create an integration test environment for the proposed 3rd party experimentation solution, we can collaborate practically with Data SRE, SRE, and QTE to evaluate the solution’s viability within WMF infrastructure in order to make a confident build/install/buy recommendation.", we will need to set up test integrations of the two options under consideration

To more effectively work together effectively, Test Kitchen will need input from Traffic - early and often.

We have prepared an initial decision brief covering licensing issues that has been submitted to senior leadership.

Our next steps will involve standing up example integration of the two options in order to fully evaluate

  1. If the specific platforms can be integrated with WMF infrastructure and, if so, what caveats or challenges exist
  2. The specific platforms are available in a state that allows for user testing based on the product requirements

Requests for Traffic

  1. Review both Growthbook and Statsig, broadly
  2. Walk through system diagrams live with Test Kitchen team
  3. Provide iterative input on system design

Acceptance Criteria

  • An example integration of Growthbook is running
  • An example integration of Statsig is running
  • The system designs covering both systems have been reviewed and signed off, or not, by Traffic

Event Timeline

@KOfori we would love to collaborate with the Traffic team as we make this decision. Is there anyone from Traffic who would be available to review the two options we are considering and give us their feedback?

CC @WDoranWMF and @BBlack

odimitrijevic claimed this task.
odimitrijevic subscribed.

I will go ahead and close this task since the POCs are complete. The proposed integration architecture introduces a proxy service that will provide experiment configurations to the wikis in the same way, regardless of the experimentation platform. When this service is proposed, we'll engage the appropriate teams for review and approval.