Page MenuHomePhabricator

[Reading Lists] Monitor potential performance impact of Reading Lists for Web
Closed, ResolvedPublic2 Estimated Story Points

Description

Background

The new Reader Experience team is planning on working on Reading Lists for the web during Q1 25-26. The Reading Lists hypothesis in the annual plan states:

If we give logged-in readers the ability to save articles to a private reading list, we expect engagement on the site to increase, as measured by a 5% increase in internal referral traffic for readers who use the feature, and a statistically significant increase for all users.

Which means we are aiming to see an increase in engagement from logged-in users. This task seeks to define metrics that could be impacted by this feature and measures the back-end performance impact of this feature if any (following up from a couple of meetings between Web and SRE earlier this year).

Potential metrics impacted by Reading Lists on web

Potential metrics to monitor

  • Impact of general increase in traffic from users using Reading Lists
  • Impact at the application level for Reading List activity
  • Increase database load due to Reading List activity

TODO

  • Connect SRE to make a plan around deployment.

Outcome

  • Create follow up tasks as needed.

Event Timeline

Jdlrobson-WMF set the point value for this task to 2.

Are there any early estimates of the expected %age increase in something like logged-in daily active users?

@SToyofuku-WMF to add charts and graphs from the DB side, courtesy of Amir

When I talked to Traffic about this topic earlier in the year, the concerns were mainly around potential increase in logged-in user traffic. Since then, the design of this first Reading Lists experiment has changed. We're planning to release the initial version of Reading Lists to a max 200,000 existing logged-in users spanning 6 wikis: enwiki, frwiki, arwiki, idwiki, viwiki, zhwiki.

Given the very limited nature of this rollout, @CDanis , we're not expecting any increase in logged-in user traffic from this specific experiment any more. We do plan to encourage users to create accounts to use features like Reading Lists in the future, and we'll work closely with Traffic & SRE to determine the scale of that future experiment, potentially some time in Q3 or Q4.

But for now, my guess is that we don't have to monitor impact on traffic at this point (but it would be great to verify this @CDanis). We've already consulted with Amir on the potential DB impact as Steph mentioned above.

Thanks for those graphs @SToyofuku-WMF , let's monitor those when we launch the experiment. Given that we're not creating any additional accounts with this experiment, that's seems sufficient.