Are these links worth instrumenting within an experiment given they do not exist for the control group?

Yes, because we want to understand how they are actually used and if we really need them. They are much harder to sensibly translate than just one single link to a wiki's privacy policy.

If so, should the outcome be part of TestKitchen's automated analysis report? Or we're using GrowthBook for all the metrics in this experiment?

Mh, good question. On the one hand, TestKitchen AutomaticAnalysis (TK AA) should be good enough. On the other hand, that would split the analysis between yet another platform. Also, having this in Growthbook should allow us to get some experience with how to use that while handling with a link CTR a relatively simple metric. So, I'd say let's go for GrowthBook here.

Change #1278379 had a related patch set uploaded (by EMcFarland; author: EMcFarland):

[mediawiki/core@master] UsernamePolicyPopover: add instrumentation for links

https://gerrit.wikimedia.org/r/1278379

Hi, I wanted to update this patch after we had a growth engineering team discussion. This work is dependent on Michael's in progress work to support non-cache splitting experiments for account creation, linked here https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/1286930. (I will update my patch's commit message to make the dependency apparent.) As a group, we agreed that the exposure event should occur once users get to the create account page, and not when users open the popover. I'll remove the send event code from my work on the popover. Finally, my patch currently uses the experiment as the third param in the methods called to start the ClickthroughRateInstrument. That won't work anymore, because of not being able to get the experiment from the promise. Instead, a new instance of the experiment class would be needed.

Also in our growth engineering meeting, we discussed if the instrumentation I worked on needs to be registered for Test Kitchen Automated Analysis. Michael's thought was, no, but we will need to change GrowthBook. We can discuss this more with Mikhail at the offsite next week.

Depends on: T424646: Prepare V2 experiment for improved mobile account creation form (builds on V1)

Change #1278379 merged by jenkins-bot:

[mediawiki/core@master] UsernamePolicyPopover: add instrumentation for links

https://gerrit.wikimedia.org/r/1278379

This should now be testable on https://auth.wikimedia.beta.wmcloud.org/enwiki/w/index.php?title=Special:CreateAccount&experiments=we-1-8-account-creation-form-v2:treatment:overridden&mobileaction=toggle_view_mobile

Change #1298766 had a related patch set uploaded (by Michael Große; author: EMcFarland):

[mediawiki/core@wmf/1.47.0-wmf.5] UsernamePolicyPopover: add instrumentation for links

https://gerrit.wikimedia.org/r/1298766

Change #1298766 merged by jenkins-bot:

[mediawiki/core@wmf/1.47.0-wmf.5] UsernamePolicyPopover: add instrumentation for links

https://gerrit.wikimedia.org/r/1298766

I have checked this on the console and clicking on the popover links fires an event with an object that has the shape below, where the element_friendly_name capturing the link that was clicked.

we-1-8-account-creation-form-v2: The enrollment for this experiment has been overridden. The following event will not be sent:
 click {
  "funnel_entry_token": "51cd4a34c2341dc051a4",
  "element_friendly_name": "username policy popover informational link - real name",
  "action": "click",
  "experiment": {
    "enrolled": "we-1-8-account-creation-form-v2",
    "assigned": "treatment",
    "subject_id": "awaiting",
    "coordinator": "default",
    "sampling_unit": "edge-unique"
  },
  "agent": {
    "client_platform_family": "mobile_browser",
    "client_platform": "mediawiki_js"
  },
  "mediawiki": {
    "database": "enwiki",
    "skin": "minerva"
  },
  "performer": {
    "is_temp": false,
    "is_logged_in": false,
    "is_bot": false
  },
  "$schema": "/analytics/product_metrics/web/base/2.0.0",
  "meta": {
    "stream": "product_metrics.web_base"
  }
}

This can be closed as resolved.