I used queries that we developed for the hCaptcha account creation A/B test analysis for a snapshot analysis, with data from the first three weeks of January 2026. We find that the data allows us to easily answer the first two questions, and that we're unfortunately unable to answer the last question, which might be the most important one to answer. I'll be filing a subtask of T394744 to update the instrumentation to fix that problem.

In T413906#11498225, @KStoller-WMF wrote:

@nettrom_WMF - does this help unblock some of your current work, or is this more of a "nice to have" that's documenting a instrumentation quirk that might cause future frustration?

Tagging this with Test Kitchen as it affects MP PHP library usage, and partly because I want to make sure this bug doesn't get lost.

In T404127#11328606, @mpopov wrote:

@nettrom_WMF: The team is just now seeing this. Is this a significant blocker for your analysis work?

In T404177#11325473, @Sfaci wrote:

This is already merged and backported so you should start seeing mediawiki_database as a populated attribute

I did some investigation into the data today, and found fourthree issues:

My investigation found that there doesn't appear to be something related to IRS in https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/tree/main/analytics_product/dags. I'll build upon an existing team's DAGs and submit a merge request.

Reassigning this to myself. I'll look into whether there's code for the DAG somewhere, but I did notice that this is not in the PA Airflow dashboard.

In T404177#11261389, @Madalina wrote:

@nettrom_WMF I have updated the task with the latest spec. Can you please check and confirm here that this is correct?

Reduction in proportion of blocks issued to IP addresses/ranges:

In T379179#11112619, @kostajh wrote:

I don't remember why we said we needed to opt out of including IP and user-agent data. I think we should leave this as is.

For future reference, I've uploaded the notebooks used for data gathering and analysis to GitLab: https://gitlab.wikimedia.org/nettrom/2025-temp-accounts

The notebook that I created to understand the extent of this is now available on Gitlab: T366222-global-local-issue-investigation.ipynb

@JJMC89 : Thank you for chiming in with information about this! Those were great points, particularly the fact that a block might just be a continuation of a previous block and therefore naturally have longer duration than previously.

For documentation purposes, here are the key topics we discussed and made decisions about:

@kostajh asked me to review the contextual attributes. I'd add agent_client_platform_family to the list, since that makes it easy to identify desktop/mobile web usage. That's all, looks good to me!

In T366222#10875457, @Astinson wrote:

One of the unintended effects of the IP Blocks is that it prevents account creation -- do we have any logs that (i.e. goes from one of these event pages, tries to create an account (thus has a return target page in the return url), but is prevented from doing so -- do we log the return url anywhere? Could we probe if the return urls are in the Event namespace?

I'll take another look at the Measurement plan tomorrow when I have more time. In the meantime I wanted to expand on a comment I left in the Instrumentation plan where I suggested we use the funnel_name and funnel_entry_token fields in the schema.

In T333221#10894559, @Niharika wrote:

@nettrom_WMF Do you think this task is still needed? Is this a blocker to any upcoming temp accounts deployments?

I tagged the Editing Team since from what I understand they created the original instrumentation in T310390, but T&SP is my touch point at the moment, so I'll be chatting with them first.

I left this open for a week in case there were questions or comments that needed addressing. Wrapping it up now as resolved.

This has been completed by rerunning the Airflow DAGs for that snapshot. Thanks to the Data Engineering ops team!

Might be easier to do this through Airflow. Waiting for the DAG errors to go away first, though.

Summary: Our analysis compared the deployment period to the same time period in two reference years (2022 and 2019). We find no significant impact of Temporary Accounts on any metric except the proportion of blocks issued that are IP-based blocks, where we find that Temporary Accounts are associated with a strong decrease. This indicates that wikis change from blocking IP addresses or ranges to blocking temporary accounts.

In T387600#10868614, @Tchanders wrote:

If that's no longer part of the tool, then I think it's easier for us to not instrument session expiry and instead infer it based on the parameters from the session start plus any "+10m" clicks (for sessions where the user doesn't explicitly turn the feature off, of course).

That sounds fine to me.

In T387600#10845757, @Tchanders wrote:

The expiry is the only one that doesn't line up with a specific line of code being called, at least at the moment of expiry.

However, we can capture after the fact that an auto-reveal session expired (rather than being switched off manually). There is a line of code that checks whether auto-reveal is on, and updates the database if the mode has expired - there's just no guarantee that this will be called very soon after expiry. So if we want to know at any given time how many sessions have expired, we'd have a lower limit (some might have expired but not been captured by our instrumentation yet).

Closing this as declined as there hasn't been a push for prioritizing this. Can be reopened and reassigned if needs change.

Thanks for answering my questions @Tchanders, it's good to get confirmations about those things! No worries that the reminder feature isn't available at launch, it's trivial to plan for it being added later.

I've taken a look at this and would like to apologize in advance about the number of questions and comments I have. They'll help me understand the context and what decisions we're trying to make, so that the instrumentation and measurements can help support those. They're also to make sure we've covered all the bases we're interested in.

In T366222#10782683, @SEgt-WMF wrote:

I was wondering: is there an opportunity to look into whether the 2% of blocked edit attempts during campaigns were geographically concentrated, or if they were more evenly distributed across regions?

I've completed an initial investigation into this, finding that 2% of users who signed up for one or more campaigns encountered at least one blocked edit attempt during those campaigns.

Assigning this to me as I'll pick this up and do a quick investigation into this.

@AUgolnikova-WMF : There's a gap in the dashboard between Dec 20222 and Feb 2025 because the dashboard was no longer in use, and because the underlying data source doesn't go back further than 90 days. The "Daily Media Searches in VE" chart is different, it'll always show the last 90 days because that's querying event data directly (and that's the same data source that used for the aggregations shown in all the other charts).

I've completed backfilling the data from Feb 1 onwards. There's now a daily cron job that updates the underlying data, it runs at 05:20 UTC every day. The dashboard has been updated with filters so that it by default shows the last quarter's data. Older data is still available, it's just not shown immediately.

I've uploaded the notebook that I used for gathering data on the usage of mw.track() to this GitLab repo.

This analysis has been completed. We found no significant difference between the group of users who saw the Community Updates module and the control group. The primary reason for this is that the number of users who registered on Spanish Wikipedia during the experiment and signed up for a campaign is too low.

Fixing subscribers.

Resolving this task as the work has been published in the Newcomer Homepage paper, presented at CSCW 2023. We also presented this work in the May 2024 Research Showcase.

Merge request submitted: https://gitlab.wikimedia.org/repos/data-engineering/wmfdata-python/-/merge_requests/65

In T374664#10267324, @Sgs wrote:

@nettrom_WMF, in our last meeting you mentioned you would define treatment and control group more precisely in terms of account ages. For now, we're not restricting any account by its age, all accounts including newly created are distributed 50/50 into control and the treatment group community-updates-module. Is this ok the rollout on Wed, October 30 to Spanish Wikipedia, French Wikipedia, and Egyptian Arabic?

Confirmed by generating events on testwiki and querying logged events in the Data Lake that performer.id is now captured correctly in the database table. Closing this as resolved, thanks everyone!

In T372840#10227052, @KStoller-WMF wrote:

I had assumed we would do a 50/50 split for both new accounts AND existing accounts as I imagine that will allow us to gather enough data to make a statistically significant conclusion the fastest. But I'm open to whatever @nettrom_WMF thinks is best.

In T374675#10158054, @nettrom_WMF wrote:

Moving this to "blocked" for now because I'll be waiting on or coordinating with folks in T371347 to verify the data in the Data Lake once we figure out where it is (currently there's only a few events from 2024-09-11).

I've verified that the impression and click events are firing as expected for the module on testwiki.

@KStoller-WMF : Sounds good to me! Ping me when the overview is on MW and I'll take a look?

The estimate has been done and the associated googledoc shared with stakeholders for review. In short, we're in need of about 10k visitors to the Homepage, which in a worst-case scenario we're expected to get in 1.5 months. We also have valid reasons to expect that worst-case scenario to not come true, but we've also defined leading indicators to tell us what is happening just in case.

I've taken a look at it and couldn't find anything to add or change. Moving it to "done". Maybe @mpopov wants to have the pleasure of closing this?

I checked out the data and the associated dashboard, and this looks good to me.

Awesome example dashboard, great work!

Confirmed that these click events are firing correctly on both desktop and mobile web, moving to "done".

When verifying these events, I don't see action_context set for these (but it's set for the subsequent click event). Moving it back to "blocked" for now, but that might not be the right column?

Looks good to me, moving to "Done"

In T374251#10140473, @Sgs wrote:

I have been highly discouraged from taking this approach by @mforns and @phuedx so I'm avoiding it for now, action_context=friendly_update_title. One caveat is we don't have anything that identifies uniquely a "community update". If the title is updated but it's an update on the same announcement, we'll get two different contexts.

While I know a decision has been made, I wanted to add a few notes about option 1 in case someone comes back to this in the future. From the current task description:

In T374464#10134507, @Sgs wrote:

Looping @nettrom_WMF to give us some guidance on which views would be more interesting to create in the context of the experiment.

In T374251#10126287, @Sgs wrote:

Thanks for providing the detailed specification @nettrom_WMF. I have a some questions:

1. In the instrument specification the field used to capture the friendly name is action_subtype, the documentation for that field says it is used to classify actions, I'm not sure if the update friendly name is a good candidate for such classification. I wonder how is it different from capturing it under action_context? I'm imaging we introduce a second link in the future (eg: wikitext is allowed in the description), how would we differentiate the source link that originated the event?

I think the last time we used this data to make changes was in 2019, where clicks on these links were one of the Help Panel Leading Indicators. My recollection, and @Trizek-WMF should correct me if I'm wrong, is that afterwards we had a fairly standard set of suggested links to go in the Help Panel (and subsequently the Help module on the Homepage) that reflected those stats.