I wrongly stated that this is only happening in idwiki because that was the only affected wiki showing in Growth's team validation dashboard. However, this is affecting several other wikis, see logs.

This won't be visible in production anymore after the revert from T422288.

After discussion with @Urbanecm_WMF and discussing the recent incident T422288: Massive performance problems apparently related to the GrowthExperiments extension, we agree that the impact module is not in a stable shape to scale its edit limits broadly.

After an audit discussion with @MNeisler around the current Account creation rate metric possible problems, we identified a potential pitfall being how the query is currently written. As it is now, there's no guarantee that the subjects for which we counted their experiment_exposure match the subjects for which we counted their successful registration via account_created event. That's because all users that try to register an account within the experiment via any link in our site, not just the anon warning, will have an enrollment attempt and potentially be part of the experiment even if they were never exposed to the anon warning.

In T416666#11782171, @Michael wrote:

I think we talked about this error-message at the top: it is fine to remain as a block-styled error message. Only the ones directly associated with an input should change to the inline style.

Ack. Also, for clarity, this change can be released as regular improvement, not platform specific, and it is not part of any create account experiment right?

HTMLForm on its "codex" variant already uses Codex field components. The first two acceptance criteria are already met by current status quo:

• Special:CreateAccount uses Codex Field components for form inputs
• Validation messages use Codex inline Message component

Maybe a fallout of T415288?

It seems that the schema for PHP configs have change to now have user_identifier_type as a required property (Coordination/EnrollmentsProcessor.php#19). fyi, I've updated TestKitchen#Testing_an_Experiment_on_Logged-In_Users accordingly for those who work with local configs instead of remote fetching. Hope that makes sense, also maybe there are other places to update that I've missed.

Discussed with @MNeisler and a plausible explanation would be that we're sending edit_saved events for anon users, and these do not contain the .performer.edit_count property , see example:
{P90096}

Once we "Account creation rate" as defined metric in the catalog this will automatically show for each experiment using that metric in Test Kitchen Metrics Overview.

The limit to 10K is now enabled in enwiki, giving some days to monitor health signals before rolling out to more wikis.

Per conversation with @MNeisler it is ok for her to work with element_id as the source of information for this, see T416739#11672889. Tentative resolution.

In T415536#11708499, @Etonkovidova wrote:

Checked for getUserVariant - https://codesearch.wmcloud.org/search/?q=getUserVariant - it is still listed as used getUserVariantForLegacySchema and in includes/Language/LanguageConverter.php. Is it ok for the scope of the task?

Per docs in CentralAuthPostLoginRedirect:

Increasing priority as this has come up already in T417876

I think feature toggling would be enough for patch demo and data collection testing can de done in beta and testwiki, but Michael can correct me.

In T416100#11690555, @KStoller-WMF wrote:

KPIs

Account Creation
TBD: Constructive Activation
TBD: Constructive Edit Rate

@Sgs In the task description, I listed Account Creation as the main KPI. This is the primary metric we will use to measure success.

I listed Constructive Activation and Constructive Edit Rate as TBD because I was not sure whether they can be measured with the current Test Kitchen setup. I see these as secondary metrics that we would ideally track. However, if tracking them would significantly complicate the experiment release, I am open to adjusting the acceptance criteria.

If we need to prioritize, Constructive Edit Rate is the lowest priority, especially if we are able to track Constructive Activation.

I found another issue coming from this task while working on the instrumentation, the create account link was missing the returnto parameter, hence breaking the so called mid edit signup flow. This seems a relevant use-case to take in account for T402719.