Change #1313989 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):

[mediawiki/extensions/FlaggedRevs@master] [WIP] Send notification for edit being accepted for newcomers

https://gerrit.wikimedia.org/r/1313989

Ohhhh, exciting! Yes, @Samwalton9-WMF and I have discussed how we would love to run an A/B test for something like this. If we need to minimize complexity, I think we should at least look at pre/post rollout data to monitor trends.

I agree it's worth making sure we can analyse the impact of this!

The only concern I have would be making sure that newcomers aren't being overwhelmed by notifications from this, if all of their edits are being marked as reviewed. How does bundling work in Echo, could we e.g. have a user log in to see "8 edits reviewed", rather than 8 individual notifications?

Yeah, for now my idea was to enable it only on wikis where protected mode is enabled so pages that require review would be a small subset but yeah, having it on German Wikipedia for example would be just annoying until you hit 100 edits.

For Wikis where all pages need review, we can do either of these two:

• Debounce: Only notify the user if they haven't been notified for any pages in the past week.
• Randomness: Flip a coin, notify them 10% of the time.

Of course it means the message should be very clear that the review is for some of their edits so they don't get confused when they don't get notification for other reviewed edits.

This could also have a form of a daily aggregated report.

Good points! My understanding is that Echo doesn't handle bundling very well right now, but I agree we need to make sure these notifications don't end up feeling spammy or overwhelming. From my perspective, I would prefer a predictable cadence, such as a daily or weekly aggregated notification, rather than sending notifications at random. Predictability helps people build an understanding of what to expect, which is especially valuable for newcomers.

My original vision was to limit these notifications to a user's first edit or first several approved edits. The primary goal is not simply to notify people every time an edit is reviewed, but to address a specific onboarding problem: newcomers who are making good edits often feel like they are "editing into a void" because they receive no feedback.

An approval notification serves two purposes. First, it provides a small but meaningful piece of positive reinforcement that can increase confidence and encourage someone to return. Second, it teaches newcomers how the moderation process works by making the review process visible. Without that feedback, many editors may never realize their contributions were reviewed and accepted.

Loving the progress and I'm more than happy to help with evaluation post-launch! Some additional data / links for context:

• Details on what currently generates notifications: https://en.wikipedia.org/wiki/Special:DisplayNotificationsConfiguration
• Unfortunately the Special:ValidationStatistics page generated by FlaggedRevs only seems to show average time-to-review for the protection-mode wikis, which is often heavily skewed by a single unreviewed edit. But some data on median time-to-review from the larger FlaggedRevs wikis shows ranges from 5 minutes to 14 days. So in most cases, newcomers would have to revisit the article they edited (presumably a few times) before they'd notice their edit taking effect. I'll see if I can dig up more complete data though.
• I keep asserting that newcomers likely aren't putting pages on their watchlist when they edit existing pages because it's not the default action. I finally actually computed some data on that and looked at the first ten edits from editors on Wikipedia who created their account in the first three months of this year (query below). Only 25% of the time was the page that they edited on their current watchlist. This data allows for folks adding the page to their watchlist well after the edit or having added it but with an expiration that has passed, but I assume that's minimal. 25% of these newcomer edits were reverted (so the person got a notification regardless of watchlist add) but that still leaves 55% of edits that were neither reverted nor added to a watchlist, so editing and presumably hearing nothing about it :(

snapshot = '2026-06'
start_date = '2026-01-01'
end_date = '2026-04-01'

df = spark.sql(f"""
WITH wikipedia_projects AS (
    SELECT DISTINCT
      database_code
    FROM canonical_data.wikis
    WHERE
      database_group = 'wikipedia'
      AND status = 'open'
      AND visibility = 'public'
      AND editability = 'public'
),
target_editors AS (
    SELECT DISTINCT
        event_user_central_id
    FROM wmf.mediawiki_history mwh
    INNER JOIN wikipedia_projects wp
      ON (mwh.wiki_db = wp.database_code)
    WHERE snapshot = '{snapshot}'
      AND event_entity = 'revision'
      AND event_type = 'create'
      AND SIZE(event_user_is_bot_by_historical) < 1
      AND SIZE(event_user_is_bot_by) < 1
      AND event_user_registration_timestamp >= '{start_date}'
      AND event_user_registration_timestamp < '{end_date}'
      AND event_user_is_created_by_self
      AND NOT SIZE(event_user_blocks) > 0
)
SELECT 
  mwh.wiki_db,
  event_user_id,
  page_title,
  event_user_revision_count,
  COALESCE(ARRAY_CONTAINS(revision_tags, 'mw-reverted'), False) AS reverted,
  COALESCE(wl.wl_user > 0, False) AS on_watchlist
FROM wmf.mediawiki_history mwh
INNER JOIN target_editors te
  ON (mwh.event_user_central_id = te.event_user_central_id)
INNER JOIN wikipedia_projects wp
  ON (mwh.wiki_db = wp.database_code)
LEFT JOIN wmf_raw.mediawiki_private_watchlist wl
  ON (mwh.wiki_db = wl.wiki_db
      AND wl.snapshot = '{snapshot}'
      AND mwh.event_user_id = wl.wl_user
      AND mwh.page_title = wl.wl_title
      AND wl.wl_namespace = 0)
WHERE
  mwh.snapshot = '{snapshot}'
  AND event_entity = 'revision'
  AND event_type = 'create'
  AND event_timestamp >= '{start_date}'
  AND event_user_revision_count <= 10
  AND page_namespace = 0
""")