Progress update on the hypothesis WE2.1.1: Week 1

Team has developed high-level selection criteria and measurement plan for growth feature enablement.

Selection criteria:

• 5 to 25 active editors
• Maintenance templates
• 1–2 admins

Metrics for Q1 evaluation

• Number of accounts created
• Number of people who used Growth features
• Number of people who clicked and viewed the banner
• Number of editors retained from the experiment X days later

The technical requirement for banner instrumentation has also been identified. A task has been created to set up tracking for users who create accounts through CentralNotice banners for WE2.1.1: T398646.

CentralNotice banner designs were created by Brand Studio team, shared by @MSRamos and others, and feedback was provided.

@KCVelaga and @cchen plan to share the initial list of wikis by July 11.

A new call to action for the central notice banner was suggested by @satdeep_gill and @FRomeo_WMF . It involves exploring the use of the EventRegistration tool as an informational landing page for people who are not logged in, where tasks are listed for them, and information is provided to guide them in using Growth features and participating in editathons or other community events (TBD).

Why are there both a project tag (with tasks tagged) and a task (with subtasks) to track the same thing?

@Nikerabbit Now that we have a subtask should we consider archiving https://phabricator.wikimedia.org/project/view/8033/? What do you think?

Progress update on the hypothesis WE2.1.1: Week 2

Initial list of 88 wikis for growth features enablement shared by @cchen for team's review. Discussions are underway on further filtering to narrow down the list. Request for Central Notice Banner and admin rights initiated at meta-wiki here.

A new requirement has arisen to code the banner in HTML/JS. We are currently investigating who can help with this.
Based on a comment by a CentralNotice admin, we may also need to consider Event Registration as the landing page: T398646#10999781.

@srishakatux https://phabricator.wikimedia.org/project/view/8033/ aims to capture currently active LPL projects please see https://phabricator.wikimedia.org/project/board/8029/ and also description https://phabricator.wikimedia.org/project/profile/7245/

In T399011#11001864, @srishakatux wrote:

@Nikerabbit Now that we have a subtask should we consider archiving https://phabricator.wikimedia.org/project/view/8033/? What do you think?

@MaryMunyoki Ahha got it! Makes sense.

Progress update on the hypothesis WE2.1.1: Week 3

A list of 10 wikis has been shared by @cchen for team's review after considering bucketing and random selection based on editors and content count.

Considerations for the event registration as an additional call to action for the banner creates an additional scope of work beyond the planned work. Other colleagues have emphasized the role of organizers to be an important one especially for smaller communities. Previous experiments with Central Notice Banners have highlighted that new communities need more persuasion to edit. Availability of organizers in the select communities have been highlighted for consideration.

Progress update on the hypothesis WE2.1.1: Week 4

Growth Features and Communication Support
We discussed the list of 10 selected wikis with @KStoller-WMF and @Trizek-WMF from the Growth-Team. It appears that all of the selected wikis have features available via the GrowthExperiments enabled, including the "Add-a-Link" model and suggested tasks, except for two lowiki and abwiki which we may consider skipping. There have also been discussions about localization efforts for Growth features on these wikis, which will be addressed by @UOzurumba during communications with the communities: T400301. The goal is to reach 75% localization coverage before the banner launch. For active communities, this might take one week to complete. We also discussed further configuration and localization of Growth features, prioritizing certain wiki groups.

Central Notice Banner and User Scenarios
There have been conversations with Central Notice admins on the Phabricator ticket and wiki page regarding banner requirements: T398646. Based on their suggestion not to direct non-registered users to the Newcomer Homepage, we discussed two possible scenarios (see screenshots attached):

Scenario A: Call to Action - Event Registration
All users are shown an event registration page that provides information on how to create an account, log in, and edit using the Newcomer tool.

Scenario B: Call to Action - Newcomer Homepage
When registered users click on a call to action, they are shown the Newcomer Homepage or a message prompting them to enable it in their user preferences. Non-registered users are directed to create an account.

A draft of the Central Notice landing page has been prepared by @Sadads.

{F65687045}
{F65687044}

Wiki Selection
Currently, the high-traffic wikis for six of the selected wikis are among the top 20 Wikipedias by page views. This may mean we need to work with more than 10 wikis, although we want to limit it to 8 wikis for the scope of work. We're exploring the possibility of narrowing down this list to include English Wikipedia. @cchen will work on adding a list of “second top wikis” (i.e., those with the next highest page views) and include page view data for both "top" and "second top" wikis in the spreadsheet.

Progress update on the hypothesis WE2.1.1: Week 5

Received approval from Central Notice admins to proceed with the experiment: https://meta.wikimedia.org/wiki/CentralNotice/Request/WMF_Language_Onboarding_and_Development_Experiment 🎉

Central Notice banner messages are ready to be requested for translation here.

Decided to downsize the number of wikis to 7, including 6 small wikis + 1 English wiki, which will serve as a high-traffic wiki for all small wikis. We are still figuring out how to get localization for Growth features completed for all selected wikis. Currently, we have no community contacts for three of them, as confirmed with the WMF-Communications team. As a backup, @cchen plan to add data around localization activity for all 88 wikis into a spreadsheet and reselect wikis that meet 75% of the criteria, followed by regrouping. We are also reviewing some data discrepancies and conducting a human review to determine which sub-national regions we should target for the small wikis, since that data is not readily available (e.g., page views).

The first draft of communication to small wikis has been prepared by @UOzurumba. We plan to reach out to the currently selected or newly selected set of small wikis by the end of next week (August 8th).

With Wikimania approaching, and localization activity being a bottleneck combined with guidance from others that small wikis need enough time to meet the requirements, we are now aiming for a banner launch on August 29. Another issue we’ve encountered is with designing Central Notice banners. @srishakatux received admin rights earlier this week but is running into an issue with banner previews that show an "edit token has expired or is missing" error. A meeting is scheduled with @Ejegg from Fundraising Tech team next week to debug this issue.

Progress update on the hypothesis WE2.1.1: Week 6

@cchen added localization activity for growth features in 88 wikis. Since localization is low for all (<75%), it makes sense to stick with the six wikis we have. Event registration as a call to action is planned with Punjabi, Minangkabau, and Moroccan Arabic, where we have contacts. Newcomer homepage as a call to action is planned for the remaining languages: Luxembourgish, Zulu, and Asturian as advised by @satdeep_gill

The technical issue related to user rights on CentralNotice, which prevented @srishakatux as a new user from interacting, was resolved with debugging help from @Ejegg in Fundraising Tech: T401143. Based on the different scenarios considered for the call to action for the banner, three banners have been created to be used on six small wikis and their corresponding high-traffic wikis:

• Event registration as a CTA for all users: https://meta.wikimedia.org/wiki/Special:CentralNoticeBanners/Edit/Language_experiment_cta_event_registration
• Newcomer homepage as a CTA for registered users (logged-in and anonymous): https://meta.wikimedia.org/wiki/Special:CentralNoticeBanners/Edit/Language_experiment_cta_newcomer_homepage
• Create account as a CTA for non-registered users: https://meta.wikimedia.org/wiki/Special:CentralNoticeBanners/Edit/Language_experiment_cta_create_account

A list of subnational regions where the banner will be displayed on both high-traffic and small wikis for six selected languages has been created. The regions reflect what is available in the Central Banner Admin interface: T391045#11071277. There will be 18 different variations of the banner that will be displayed across six different wikis, each featuring a different call to action (CTA). Registered and non-registered users will see different versions: T391045#11071277.

@UOzurumba is continuously providing communications support to small wikis. More update on this will be shared in the next report. @Sadads added this draft event language experiment page on meta wiki to be used as a reference by admins of small wikis: https://meta.wikimedia.org/wiki/Event:DRAFT_EVENT_LANGUAGE_EXPERIMENT

Progress update on the hypothesis WE2.1.1: Week 7

Small and large wikis were contacted by @UOzurumba to obtain approval and support with completing prerequisites for the experiment:

• Minangkabau
• Moroccan Arabic
• Punjabi
• Zulu
• Luxembourgish
• Asturian
• Spanish
• English

Conversations with admins of these wikis are ongoing, and admins of all small wikis have been reached.

Minor changes to the banner background color were made after feedback from @Sadads and the Fundraising team on which colors perform better. Banners were also adjusted to work reasonably well in dark mode, which required some technical investigation.

Campaign creation for each targeted region/wiki has started, beginning with the English Wikipedia banner in the Spain region. In total, 18 variations will be set up over the next two weeks.

• WMF_LOD_CA_EN_ES campaign
• WMF_LOD_CA_EN_ES banner

Punjabi Wikipedia Event Page is created.

The only blocker at the moment is if Moroccan Arabic and Minangkabau contributors are unable to finish creating their event landing pages before the deadline, although admins of those wikis are in touch.
@hueitan is looking into setting up tracking for the banner, which will hopefully be completed by next week.
A meeting with @Trizek-WMF is planned to do a final check on user scenarios (registered / non-registered users) for growth features.

Progress update on the hypothesis WE2.1.1: Week 8

Measurement plan has been developed and shared by @cchen

Moroccan Arabic and Minangkabau, the only two wikis that were still missing event pages, have now added them on their wiki.

Tracking links for two banner types have been created by @hueitan. For the third banner type, which uses event registration as the CTA, an additional requirement has emerged. Tracking requires code changes to certain repositories, including the Campaigns Extension. Learn more in these tickets:
https://phabricator.wikimedia.org/T402496
https://phabricator.wikimedia.org/T402497
https://phabricator.wikimedia.org/T402498.

These changes are currently pending code review. If the code review for the tracking-related changes is completed this week, the banner will be launched on Friday, August 29. If not, the launch may be delayed.

Progress update on the hypothesis WE2.1.1: Week 9

This week, work continued on setting up banner tracking, which also involved updating code related to schema changes in the Growth Experiments extension. A few CI errors needed to be addressed, and the changes are now ready for merge and deployment:

• https://gitlab.wikimedia.org/repos/data-engineering/schemas-event-secondary/-/merge_requests/75
• https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/1181699

A backport will be done on Monday to deploy these changes, aiming to have the banner ready for setup by Monday, September 1st. The new date is September 3rd for the banner launch, and all communities have been informed of this update.

Progress update on the hypothesis WE2.1.1: Week 10

This week we reached an important milestone in the hypothesis. All 18 Central Notice banners with 3 call-to-actions are now enabled, reaching 8 wikis across 11 countries and 43 regions: https://phabricator.wikimedia.org/T391045#11071277 🎉

To recap from previous reports, the goal of this campaign is to leverage high-traffic wikis to increase awareness about smaller wikis and attract native speakers from these regions to use two newcomer-targeted tools: Event Registration and the Newcomer Homepage, in order to improve vital content on these wikis.

Here are some example banners:

• https://en.wikipedia.org/wiki/Main_Page?banner=WMF_LOD_ER_EN_IN
• https://en.wikipedia.org/wiki/Main_Page?banner=WMF_LOD_NH_EN_LU
• https://en.wikipedia.org/wiki/Main_Page?banner=WMF_LOD_CA_AST_ES

So far, we’ve noticed 70 event registrations from Punjabi, Minangkabau, and Moroccan Arabic wikis. Users who have signed up range from new to intermediate editors.

• https://pa.wikipedia.org/wiki/Event:Language_Experiment_2025
• https://ary.wikipedia.org/wiki/فعالية:نضم لينا!
• http://min.wikipedia.org/wiki/Acara:RANCANGAN_TES_ACARA

Total banner impressions to date are 1.6 million across all campaigns, and page views for event landing pages have increased, with a daily average of 3,536 on Punjabi Wikipedia, for example, since the launch of the banner campaign.

Lessons Learned
Although, the enablement process for banners has been successfully completed, whether a user sees a banner in their region depends on a few factors. For example, on the Punjabi wiki in the Punjab (PB) region, banner allocation for our campaign is 50% because Wiki Loves Monuments is also running. In contrast, in the West Bengal region, allocation is 33.33% since there are three active campaigns, including ours. Other factors can also prevent a banner from appearing, such as user preferences that hide certain types of banners, ours is related to article writing. Banner allocation details for Punjabi Wikipedia in the Punjab region:
https://meta.wikimedia.org/wiki/Special:BannerAllocation?project=wikipedia&language=pa&country=IN&region=PB

From a couple of communities we received feedback about modifying banner messaging. They felt that mixing English and non-English text in the same sentence is not aesthetically pleasing. We have passed this feedback on to Comms, fixed the messages for the two communities: Luxembourgish (discussion link) and Moroccan Arabic wiki and in the future, another alternative could be tried in such scenarios, with modified messages for these communities.

Progress update on the hypothesis WE2.1.1: Week 11

Presented hypothesis idea and early impressions from the banner launch at this week’s internal Product & Tech meeting. Link to Slides.

One week into the banner launch, we are now coordinating the follow-up announcement with participants who registered via Event Registration tool from the three wikis: Punjabi, Minangkabau, and Moroccan Arabic. Admins are translating the follow-up email, there are nearly 90 attendees, and @UOzurumba will work with the admins to send the emails to all participants.

Moroccan Arabic community requested the banner be displayed on the French Wikipedia, as it is the highest-traffic wiki for Moroccan Arabic. Listing English as the top wiki for the language was likely an error on our part. We are correcting this, and @UOzurumba is coordinating with the French community to launch the banner from September 17–22. Whether these results will be included in the experiment evaluation will be revisited later.

Progress update on the hypothesis WE2.1.1: Week 12

Follow-up emails were sent to participants who registered for the campaign via the Event Registration tool. The banners and campaigns officially ended this week, and all of them have now been disabled. We are currently analyzing the results and expect to share them by the end of this month.

Progress update on the hypothesis WE2.1.1: Week 13

The first draft of the WE 2.1.1 report has been published by @cchen here: link. This report focuses on short-term measurements; longer-term effects will be measured three months later. For a short summary, see the comment in the Phabricator task here: link.

Some noticing/observations so far:

• Across both scenarios, there were a total of 19,076,386 banner impressions, 10,766 clicks, 233 accounts created, 128 users engaged with growth features, and 55 edits made by new editors.
• Overall, Scenario 2 (user → newcomer homepage) had a higher growth engagement rate of 1.80% compared to Scenario 1 (user → event registration → newcomer homepage), which had an engagement rate of 0.99%.
• While Scenario 2 shows a higher growth engagement rate, Scenario 1 shows more account creations and edits. This might relate to the regions targeted and the larger population of speakers in Scenario 1, whereas Scenario 2’s higher engagement rate could result from the user experience of being taken directly to the newcomer homepage rather than via the event registration page.
• Medium activity language & size comparison (minwiki & astwiki): minwiki had an engagement rate of 0.36%, while astwiki had an engagement rate of 4.18%. The difference is statistically significant. Scenario 2 shows higher growth feature homepage engagement than Scenario 1 for this group. This is particularly interesting because Minangkabau is spoken by far more people than Asturian, yet Asturian shows higher growth feature homepage engagement in Scenario 2.
• Looking at how these results compare with previous campaigns, we can reference a similar report: link.
  • For impressions, 0.001% led to account creations (compared to 0.006% in that campaign).
  • In that campaign, 3.7% of users who clicked the banner on the landing page created accounts. In our campaign, Scenario 1 saw 2.9% of landing page users create accounts, and Scenario 2 saw 2.6%.

Final report

Was this hypothesis supported or contradicted? Why?

This hypothesis was supported.

Early data across both scenarios showed 19,076,386 banner impressions, 10,766 clicks, 233 accounts created, 128 users engaged with Growth features, and 55 edits made by new editors. The campaign successfully reached a large number of users, but numbers dropped at each subsequent step, which is likely typical for smaller wikis. For context, these wikis typically have fewer than 5,000 monthly non-bot edits and under 25 monthly active editors.
Comparing these results to a previous campaign (link):

• Impression-to-account creation rate: 0.001% in this campaign vs. 0.006% previously.
• Landing-page-to-account creation rate: 2.9% in Scenario 1, 2.6% in Scenario 2, vs. 3.7% in the previous campaign.

Although conversion rates were lower, this may be attributed to several factors. The previous campaign targeted larger wikis across more regions. In this campaign many users from bigger wikis may not have been significant native speakers of the smaller wikis. Banner visibility also varied due to overlapping campaigns for certain communities, and Growth features are not enabled by default for experienced or auto-created accounts.

The results highlighted a clear gap between initial awareness and actual engagement activity. To better understand these results, comparison of this data with existing monthly trends (e.g., new account creation, monthly edits from new editors) and editor retention will be tracked over the next three months, which will help interpret these results more fully.

Scenario 1 and Scenario 2 produced comparable outcomes, making it hard to favor one approach going forward. Scenario 2 (user → newcomer homepage) had a higher Growth engagement rate (1.80%) compared to Scenario 1 (user → event registration → newcomer homepage) at 0.99%. However, Scenario 1 led to more account creations and edits. This difference may reflect the larger speaker population and regions targeted in Scenario 1, while Scenario 2’s higher engagement rate could be due to the smoother experience of going directly to the newcomer homepage rather than via event registration.

Even though localization of Growth features was not an objective and only a pre-requisite for this experiment, it was encouraging to see increased localization activity in certain communities (e.g., Zulu interface localization of Growth features rose from 0% → 55%).

Briefly describe what was accomplished over the course of the hypothesis work (list of deliverables, links to documents, etc.)

Experiment evaluation report:

• https://nbviewer.org/urls/gitlab.wikimedia.org/conniecc1/we2.1.1-kr-2025/-/raw/main/central-notification-banner-PART1.ipynb,
• https://phabricator.wikimedia.org/T403499

Banner launched in 7 countries, 9 wikis, and 43 subnational regions listed here:

• https://phabricator.wikimedia.org/T391045
• https://phabricator.wikimedia.org/T398646
• https://en.wikipedia.org/wiki/Main_Page?banner=WMF_LOD_ER_EN_IN (example)

Note: French wiki was excluded from the evaluation. We redeployed the banner on it for the Morocco region after the experiment had officially ended, as we realized it had been mistakenly left out of that region’s list of high-traffic wikis earlier.

Request for approval from Central Notice Admins:

• https://meta.wikimedia.org/wiki/CentralNotice/Request/WMF_Language_Onboarding_and_Development_Experiment

Campaign setup:

• https://pa.wikipedia.org/wiki/Event:Language_Experiment_2025
• https://ary.wikipedia.org/wiki/فعالية:نضم لينا!\ لينا!
• http://min.wikipedia.org/wiki/Acara:RANCANGAN_TES_ACARA

What are the key lessons from this hypothesis? How do those lessons contribute to this KR and guide next steps?

Banner design lessons:

• Some communities suggested improving banner messaging and avoiding mixing English and local language in the same message. The banner was modified based on requests from the Moroccan Arabic and Luxembourgish communities. Future iterations could use multiple banner variations to reach different language speakers and enhance multilingual messaging (example).
• Continue using both Event Registration and direct-to-tool approaches, as both produced similar results in the previous campaign.

Future ideas:
In future iterations, there could be a focus on exploring how to provide tailored, guided onboarding to identify scalable approaches for language onboarding for:

• Users who created accounts in the previous experiment, and
• Small wikis more broadly, learning from them where they are in their journey and which tools they need next (e.g., article expansion, easier edits, quality control, patrolling) to advance their wiki growth.

This experiment has shown that through targeted regional and language outreach, it is possible to draw native speakers to smaller wikis. However, for attracting and engaging them at scale and sustainably, it is necessary to explore solutions that:

• Weave key features into our products to increase content awareness and discovery for small language wikis.
• Emphasize staged onboarding or milestone-based guidance through a Starter Kit, starting with core usability tools, progressing to article expansion, and eventually introducing quality monitoring tools, integrating this guidance as part of the onboarding experience for new wikis beginning their journey in Wikimedia.