Jennifer and Olga were planning to meet yesterday (Monday 26th). Anything to update on this ticket here?
Clarity on this ticket would help us unblock work on T372780.

In T371238#10096826, @Jdlrobson wrote:

Jennifer and Olga were planning to meet yesterday (Monday 26th). Anything to update on this ticket here?
Clarity on this ticket would help us unblock work on T372780.

@Jdlrobson - just to clarify, this ticket is for gathering data only, specifically for measuring the current usage of related pages. The relevant ticket for metrics for the search UI recommendations experiment (the browser extension) is T371236: Identify metrics for search UI recommendations experiment

The Search platform team does track Related Pages in the following graph
Related Articles pageview actors as portion of all pageview actors; Plus clicks-to-impressions

These metrics are based on queries to the discovery.webrequest table, which tracks both the number of "morelike" API requests and pageviews that include the &wprov=rarw1 parameter. The wprov parameter is added to each Related Articles link. While this metric does not directly track actual click events, it does reflect the outcome of a click on a Related Articles link.

@Jdrewniak , what is morelike API request? what action triggers such a request? For context, I am looking for the "impression" events to estimate the click through rate of Related Articles link.

hi @jwang the "morelike API request" uses the webrequest table to track requests to an API like this:
https://en.wikipedia.org/w/api.php?action=query&list=search&srsearch=morelike:radium&srlimit=3&srprop=size&formatversion=2

Currently, this API is only used in the RelatedArticles extension to show related articles at bottom of the page, on mobile (screenshot below), so essentially, this metric seeks to track the number of relatedArticles impressions (each impression is one API request).

Daily clickthrough rate for the Related Pages feature on mobile

• Based on the data between 2024-08-09 and 2024-09-08: around 5.96% of Related Articles presentations result in a click.

• Daily trend

Note: The clickthrough rate is lower than reality. As the users might not actually see the related page links. More details are documented in superset dashboard presented by search platform team.

Daily overall percentage of internal referrals generated by the Related Pages feature on mobile

• Based on the data between 2024-08-09 and 2024-09-08: around 4.20% of internal pageviews are generated by the Related Pages feature on mobile

• Daily trend

Based on the discussion with @Jdlrobson on slack, it would be helpful to have the daily clickthrough rate for the Related Pages feature on mobile and desktop on English Wikivoyage. Here is the data.

Daily trend of CTR on English Wikivoyage

Summary:
On English Wikivoyage, mobile web has a higher click through rate (CTR).
Daily average CTR on desktop on English Wikivoyage: 6.34%
Daily average CTR on mobile web on English Wikivoyage: 9.93%

Superset link: https://superset.wikimedia.org/superset/explore/p/OjRbj9mXNZ0/

Thank you @jwang! @Jdlrobson, @jwang - curious about how we're measuring impressions here. Which of the following is an impression in this context:

1. All pageviews
2. All times when related pages is shown (measured by whether someone scrolls to the bottom of the page?)
3. Something else?

@Jdlrobson - also curious why we looked at wikivoyage specifically - is this related to our upcoming experiment in any way?

@Jdlrobson - also curious why we looked at wikivoyage specifically - is this related to our upcoming experiment in any way?

I suggested this as Wikivoyage is the only project where RelatedArticles is enabled on desktop and I felt this data point might be useful given our experiments are on desktop and this analysis on Wikipedia's is mobile only.