Page MenuHomePhabricator

Add ro.wiki to Wikipedia Library notification Superset dashboard
Closed, ResolvedPublic

Description

We have a dashboard for tracking notifications sent by the TheWikipediaLibrary extension, notifying users that they qualify to receive access to The Wikipedia Library. Although we got feedback from users on this wording when initially deploying it, we never did any A/B testing of different messaging.

On ro.wiki, @Strainu would like to test some different messages for this notification, to see if they impact clickthrough rates. This can be done by volunteers by updating/overriding translation strings. We don't measure clickthrough rates, but we do measure what % of sent notifications are marked as read, which is probably a good enough proxy.

We have an internal Superset dashboard which tracks this data, but it's only configured to track data from English Wikipedia.

Screenshot 2025-10-16 at 07.37.17.png (1×2 px, 484 KB)

If we added Romanian Wikipedia, we could help Strainu with this experiment and share data from this dashboard.

Event Timeline

A tangential question that needs answering for this experiment is "How many users should be presented with a given wording to ensure statistical significance?"

I have updated the dashboard to include rowiki.

Thanks KC!

@Strainu Over the past 12 months, 23 notifications have been sent on ro.wiki (compared to 4,830 on en.wiki), with a 'marked as read' percentage of 48%.

This is such a low number that unfortunately I think it would take an unreasonable amount of time to get a statistically significant result to wording changes, given the change we might expect to see.

That is a pity! Thanks all for your help, it was worth a shot. :) Maybe you'll find more active communities which would be interested in experimenting with this.

Closing this out, but I filed T408372 for the broader idea, in case we get time to experiment. Thanks for the idea!