Background

This is the second experiment in the Event Pathways initiative, following Test A (worklist-based matching). Where Test A triggers the post-edit invitation when an editor edits an article that appears directly on an event organizer's worklist, Test B explores a broader matching signal: topical overlap between the article being edited and an upcoming event.

Why a separate experiment for topic-based matching? Worklist matching is precise but limited in reach: it only fires when an organizer has explicitly added an article to their worklist. Topic-based matching could reach a much larger surface area of edits. This experiment tests whether topical matching can drive meaningful engagement, specifically among editors who are less likely to already know about events and more likely to find an invitation genuinely useful.

Testable hypothesis

If we invite editors to an event that matches the topic of the article they just edited, it will increase event registration rate because editors might not know about upcoming events that are aligned with their topical interests so inviting them to those should make them more likely register for an event.

Experiment details

Overview

• Experiment Name: Topic-matched Event Invitations
• Tracking Key: de-1-2-event-pathways-topic-matching
• Risk Level: Tier 3: Low Risk
• Contextual attributes: None

Traffic

• Assign Variation by Attribute: MW User ID
• Traffic Percent: 100%
• Variations
  • Control (control): editors see no dialog after publishing an edit if their edit has a topical match
  • Invited (invited): editors see the post-edit event invitation dialog when their edit meets the trigger conditions above
• Targeting
  • MediaWiki Database Code in:
    • TBD
• Expected start date: TBD

Expected duration: The 2-week window is intended to capture a meaningful volume of qualifying edits while limiting exposure during an early test of topic-based matching. Will also depend on when the experiment reaches enough statistical power to ensure reliability of results. See https://wikitech.wikimedia.org/wiki/Test_Kitchen/GrowthBook_user_guide#Know_when_to_stop_the_experiment

Eligibility conditions

• The editor has fewer than 100 lifetime edits (newcomer or junior editor)
• The wiki they edited on has an event starting within the next 14 days
• The relevant event has a topical match to the article they just edited
• The editor is not already registered for the matched event

Metrics

• Goal metrics
  • Event registration rate (https://growthbook.wikimedia.org/fact-metrics/fact__2CcviPDj292g26Z5QAyDMH)
• Secondary metrics
  • Median Event details page visits (https://growthbook.wikimedia.org/fact-metrics/fact__2CcviCeE7WujsB4jSSnviq)
  • Event invitation clickthrough rate (https://growthbook.wikimedia.org/fact-metrics/fact__2Ccvj8acpmspQMtBPz9M4D)
• Guardrail metrics [metrics you don't want to worsen]
  • None

Notes

Out of scope:

• Contribution rate – does the editor make at least one edit associated with the event via Collaborative Contributions?