Investigate a potential Sample Ratio Mismatch in the Mobile Account Menu experiment. Validate whether traffic allocation aligns with the expected experiment split and identify any instrumentation, logging, or platform-related issues that could explain the discrepancy.
Background
Initial analysis suggests a Sample Ratio Mismatch (SRM) between the control and treatment.
Current experiment data when filtered to "WE 1.8 FY25-26 Mobile Account Menu" and "Logged-out only":
This may indicate issues with experiment setup, event logging, or downstream data processing. Resolving SRM is required before metrics can be considered reliable.
Acceptance Criteria
- Review instrumentation, experiment configuration, and intended traffic split
- Growth engineering collaborates with Product Analytics and Experiment Platform teams as needed to validate experiment setup and assignment logic
- Root cause of Sample Ratio Mismatch is identified and documented
- Any issues with instrumentation, experiment setup, or data pipelines are resolved or have clear follow-up tasks
- Experiment is paused/reset if needed
