Regarding the instrumentation QA, as of today at 15h UTC the experiment is enabled in testwiki and the events can be inspected through the browser network tab or using the EventStreams app for the ones produced in the backend.

We're currently waiting on the changes from T411104 to get merged that will help with the real migration operation in all wikis (T409170).

Why are we using the "compute-schedule-compute-again" approach? If we want the computation to happen within the job, wouldn't it make more sense to just schedule the job and let it compute?

I believe the reason is an intent to make the data generated from a web request and the data generated from the refreshUserImpactData maintenance script (T324675) more consistent. In 881414: Process more articles when fetching page view data two methods are introduced, one for requesting the page view data from a web request content and another from a job context. The job one attempts consecutive calls to page views service for 5min until it fetches up to 1k articles page view data.

In T405074#11454316, @phuedx wrote:

@Sgs: In T410896: Expose Experiment Enrollment Sampling method, we introduced MediaWiki\Extension\MetricsPlatform\XLab\Coordinator#getAssignmentForUser( string $identifier, string $experimentName ). Could your custom code in GrowthExperiments be rewritten using that?

Could you elaborate on the string $identifier parameter? In the test I can see $identifier = '0x0ff1c3'; but I don't know which are the responsibilities/assumptions callers should take care. The first surprise is the type string as my current shallow understanding is we'd prefer to work with MW user local or central IDs.

In T410167#11445856, @Iflorez wrote:

@Sgs and @Michael
Live testing on Test wiki while reviewing Console event data, shows discrepancies.

I don't think the page makes much sense if SE are not enabled. Would it be fine to conditionally register it?

In T405074#11371617, @JVanderhoop-WMF wrote:

We'll keep this task as a record of the challenges here. T409965: Enable experiment enrollment in the MediaWiki Action API is addressing the use case of Experiment assigned group retrieval on API calls, and we will create a new task to target Experiment assigned group retrieval on account creation.

Not sure why mwscript-k8s is stopping at afwiki when I run this:

The optimizations are now testable in beta, cc @AAlhazwani-WMF, feel free to resolve the task if you are satisfied with the design review.

This is my proposal to capture all possible on-boarding end interactions and context:

In T407431#11389531, @KStoller-WMF wrote:

I would love to have access to a dashboard for monitoring notification engagement!
But sounds like we should not include that within the scope of this task? I added: T410544: Dashboard for tracking CTR for Growth's "48 hour" newcomer notifications

I cannot see data that tracks the correctness of the user's answers, is that something we should/could add to the click event that we track when the user clicks on "Get started"?

This sounds good given we're re-testing bucketing as-well while testing the instrumentation T405177.

Resolving based on QA feedback.

Should we keep tracking the notifications CTRs (or maybe a simplified version with just an aggregated CTR including primary and secondary links)? The development cost is almost none, building the dashboard seems the more time consuming. cc @KStoller-WMF

I don't think this task requires of any QA, resolving.

Reviewing the Figma spec, we've noticed that the border radius is not rounded for the rest of modules in the Mentor Dashboard. Is it intentional? From an engineering pov we would prefer a consistent solution where all of the borders have the same aspect.

Moving this to QA while I self-QA it and get @Iflorez's feedback.

I found an issue in the migration script while running it on dry-mode, for instance for eswiki this was the output:

There are changes:
Additions:
{
    "Mentors": {
        "5471712": {
            "awayTimestamp": "20251118130934"
        },
        "1473120": {
            "awayTimestamp": "20260623200305"
        }
    }
}
Deletions:
{
    "Mentors": {
        "5471712": {
            "awayTimestamp": "2025-11-18T13:09:34Z"
        }
    }
}

I forgot to update the script in T406701 and now it would save the values again using unix format rather than ISO. Providing a fix before running.

In T405177#11369242, @Iflorez wrote:

Hola @Sgs:

• Constructive edit rate: the spec for this event says edit (made on mobile web within a user's first 24 hours [where the edit is not reverted within 48 hrs]), I am assuming this restrictions will be ensured in post analysis rather than by the instrumentation itself, because I'm seeing time criteria applied in the constructive edit rate metric query, for instance INTERVAL 48h, is this assumption correct?

Yes, the Constructive Edit rate metric will be calculated in the analysis/query. In analysis we can pull all edit_save events and determine whether an edit qualifies as a constructive edit, based on timing rules and revert status. To calculate Constructive Edit rate we need interaction data including edit_saved.

In T407362#11358478, @AAlhazwani-WMF wrote:

@Iflorez, I have some questions about the instrumentation spec:

• Constructive edit rate: the spec for this event says edit (made on mobile web within a user's first 24 hours [where the edit is not reverted within 48 hrs]), I am assuming this restrictions will be ensured in post analysis rather than by the instrumentation itself, because I'm seeing time criteria applied in the constructive edit rate metric query, for instance INTERVAL 48h, is this assumption correct?
• Is the assumption above also correct for the rest of metrics that use the edit_save event data? Or said differently, the edit_save event needs to be recorded for all kinds of edits performed by users in the experiment sample (both groups) and during the experiment. Regardless of their account age, edit count, etc.
• The task rejection rate requires to record a page-visited event, however the task completion rate requires a Click on the RT card itself which launches the onboarding event. Is this intentional? Shouldn't the denominator for these two rates be the same one: clicks on the RT card? If not, could you clarify which "page visited" are we supposed to record?

In T405177#11336794, @Urbanecm_WMF wrote:

FWIW, the homepagevisit and homepagemodule schemas (and consequently, newcomertasks, which is linked to the previous one) currently collect client IP address. This is something we opted into (a long time ago). In addition to this, all schemas log the User-Agent. Both User-Agent and IP address are considered personal information by our privacy policy. Since we also collect unhashed user ID, we would very likely fail the last low risk criteria, which would make the collecting a Medium risk => requiring L3SC approval.

@Michael said we do not need neither UA nor IP address, so it might make sense to remove that instead, but currently, this is something we need to work against.

As I understand the comment it is assuming that we would be using these three schemas (homepagevisit, homepagemodule,newcomertasks), but it was requested from @Iflorez to use Test Kitchen as much as possible. So my idea was to use web base schema instead to ensure we remain on a low risk tier. Does this make sense?

In T407362#11358224, @AAlhazwani-WMF wrote:

+1! @Sgs would it be possible to display the button and the checkbox only on step 1?

I spotted a couple of issues with number format while testing the dialog in enwiki forcing the language to arabic using uselang=ar:

In T405176#11333834, @Dyolf77_WMF wrote:

@Dyolf77_WMF - Do you feel comfortable setting up a testable article on beta-arwiki?

Yes, I do. But I can't access the beta wiki.

This is very helpful @matmarex, thank you! I will dive into the details to try to give a rough plan and estimation to each of the proposed interventions.

In T293454#11299546, @AAlhazwani-WMF wrote:

as we work on this, we'd also like to clean up a few things:

• the i button jumps around at different viewports (it happens both with the vector and the minerva skin)
• we can remove the text "See your mentees below. Use the filters [...]" the UI is self explanatory making this copy redundant
• optimize spacing and layout (see specs on figma or below)

I think the particular text and interface we would want to modify is created in includes/EditPage/EditPage.php#2892, assuming it is ok for Growth engineers to work on that area of the code, I don't know how we'd conduct an experiment for a core feature. Being xLab/MetricsPlatform an extension the setup described in the docs https://wikitech.wikimedia.org/wiki/Experimentation_Lab/Conduct_an_experiment#Code is not directly actionable. Has xLab been used for an A/B test in core before? I vaguely remember Sam mentioning a PHP version A/B test setup, but Idk if that example is even relevant to what we want to do here. cc @mpopov @phuedx

I run it manually and it finished correctly, tentatively resolving,