Page MenuHomePhabricator

Estimate of Event Pathways trigger volume based on worklist coverage (DE1.2)
Closed, ResolvedPublic

Description

Please refer to https://www.mediawiki.org/wiki/Product_Analytics#How_to_get_help_with_data_or_analysis for help answering these prompts

What team/program is this request for?
Connection Team
What are you requesting?
An estimate of the number of Event Pathways dialog triggers we can expect. Specifically: given a set of events and the articles on each event's worklist, how many edits (and/or unique editors) on those articles are likely to occur within a 1-2 month window, since each such edit is a potential trigger for the post-edit dialog. If possible, break this down as number of events x number of articles per event x expected edit volume per article, so we can see how the trigger count scales as more worklists are created.

What is the problem you're trying to solve?
Event Pathways only prompts an editor to join an event if the article they just edited happens to be on that event's worklist. Before investing further in worklist-only matching (a deliberate choice over broader topic matching, which was deemed too noisy), we want to understand the actual scale of dialog triggers this approach will generate, to know whether it's a meaningful mechanism or too rare to matter.

What decision will you make or action will you take with the deliverable?
This will inform whether worklist-only matching can realistically drive the DE1.2 target on its own, or whether we need to prioritize additional discovery mechanisms (e.g. Worklist Stage 2 quality scoring, ML-based article suggestions) sooner to increase trigger volume.

Additional details

Event Timeline

Hi @OBenhmida,
I completed an initial analysis estimating the volume of Event Pathways dialog triggers we can expect based on the editing activity across event worklist articles.

Please review the summary below. Let me know if you have any questions or if any additional data points that would be useful to inform the decision.

Methodology

Collected event and worklist data from the ce_worklist_articles table. Joined with mediawiki_history to evaluate revision activity on articles across those worklists during a 30-day snapshot in June 2026.


Results

There are 88 event worklists currently identified in ce_worklist_articles. Worklist sizes range from a minimum of 1 article to a maximum of 300 articles, with an overall average of ~105 articles per worklist.

Overall Baseline & Distribution (88 Worklists)
Provides the average and median number of articles, 30-day edits, and 30-day distinct editors across all 88 active worklists.

MetricAverageMedian
Articles per Worklist105.0519
Edit volume over 30 days per worklist413.3031
Unique editors over 30 days per worklist155.267

Across all metrics, overall averages are 5x to 22x higher than medians. This extreme right-skew indicates that overall averages are heavily inflated by a small subset of large worklists (201+ articles) and/or individual articles with a lot of editing activity.

Because of this skew, the median is much more representative of typical event worklist activity.


Worklist Size Tier Breakdown
This table categorizes the 88 worklists into size tiers based on article count to show how article counts impact the estimated trigger volume.

Size TierNumber of articles per worklistNumber of Events% of Events30-Day Edit Volume (Average)30-day Edit Volume (Median)30-Day Unique Editors (Median)60-Day Projection (Median Edits)
Small1–15 articles3843.18%58.24.02.08.0
Medium16–50 articles1213.64%17.02.01.04.0
Large51–200 articles1213.64%145.211.04.022.0
Very Large201+ articles2629.55%1,238.9167.063.0334.0

The estimated volume of event pathways dialog triggers depends significantly on the number of articles in each worklist. More worklists will not scale volume if the number of articles on the worklist are small or if they don't include hyper-active articles. Trigger volume scales primarily as worklists with a large number of articles are created.

  • Over 56% of worklists have 50 or fewer articles. Over 60 days, a typical Small or Medium worklist generates only ~4 to 8 total edits and reaches ~2 to 4 unique editors.
  • Over 43% of worklists have more than 50 articles. Over 60 days, activity scales up significantly—ranging from a median of 22 edits (Large) to 334 edits (Very Large), reaching 4 to 63 unique editors.

Thanks @MNeisler, this is really helpful. Let me say back what I'm taking away so I know I've got it right:

Most of the volume comes from a few very big worklists. That's why the averages are so much higher than the medians. The median is the better number for a typical worklist: 19 articles, about 31 edits a month, reaching about 7 editors.

The tier breakdown is the part that matters for our decision. More than half of worklists (56%) have 50 or fewer articles, and those barely generate any edits. So just having more worklists won't give us more dialog triggers. The volume comes from the big worklists, or ones that happen to include a few very active articles. That tells me worklist matching works well when organizers build large lists, but it'll be thin for most events.

So this pushes us toward the things that add more coverage per worklist (Stage 2 quality scoring, ML article suggestions) rather than counting on worklist numbers alone. Let me know if I've got any of that wrong.

One question before I share these numbers around: in the tier table, the Medium tier (16 to 50 articles) has a lower median edit volume (2.0) than the Small tier (4.0). I'd expect it to go up as articles go up, not down. Is that a real pattern, or just because there are only 12 events in that tier? Want to make sure I describe it right if it comes up.

@OBenhmida Yes, your summary and takeways are correct.

Re the small vs medium tier anomaly, there's a combination of factors that might lead to this:
(1) Small Sample Size: As you mentioned, the difference here is mainly attributable to sample size. With only 12 events in the Medium tier, the median is highly sensitive. A difference between 2 edits/month and 4 edits/month is functionally in the noise—both mean less than 1 edit per week across the entire worklist.
(2) Event dynamics: It's also possible that at events with small worklists, organizers more actively drive participant attention to those few specific pages.

Thanks @MNeisler, that clears it up. Good point that both tiers are functionally in the noise either way, less than one edit a week across the whole worklist. The organizer attention theory for small worklists is interesting too, worth keeping in mind when we talk to organizers about worklist size.

This gives us what we needed to sequence the discovery work. Marking as resolved, thanks for turning it around quickly.