@Snwachukwu performed a coverage analysis of wprov see here https://docs.google.com/document/d/1TM3-qT-3gVh0cX6ktDH55guQzlAMca8TjP39KnEOStw/edit?tab=t.0#heading=h.9gu6xzm88rxl

After discussing with @YLiou_WMF, let's exclude the survey-based impact measurement for this one. The data would be too sparse and measuring the effect through surveys impossible. Surveys can be used if we want to set realistic targets for interventions (answering questions like: how likely are people to click on a Wikipedia link after seeing attribution?).

@Miriam happy to do a working meeting next week if you can find a mutually available time in case of any questions. I believe @JMoore-WMF may also be coordinating another working meeting, so if you and Justin can combine efforts here with @Maryana so we can put our minds together synchronously I think that's best (I have limited availability, but am happy to do 45-60 minutes next week across a session or two). Also happy to interface one-to-one on Meet (just schedule the time) if a coordinated mutually available time isn't possible for everyone.

Sandra's ideas in the Doc are a good. The Sec-Fetch-* headers should be logged into X-Analytics, ideally, so that they're trivially obtainable by any data inclined person.

For people wanting to test whether links and their clicks and their embeds and Ajax calls and native HTTP calls ... and on-and-on ... are conveying the Sec-Fetch-* headers in the way they think they should, it can be a little tricky unless one sets up one's own intercepting proxy (and some apps and platforms defend robustly against that sort of thing at all). Although, if the Sec-Fetch-* allowlisted headers set would be permitted to flow into X-Analytics then it should become possible in nearer realtime with a kafkacat, or for those without that access they could check on wmf.webrequest later for their navigations (or both) from Jupyter.

The behavior for what values get sent when the user navigates by clicking a link that opens from an app into a full blown separate web browser versus an app into an embedded webview (and what happens when they click on another link from within the webview) anecdotally varies; I haven't done a full spelunking on that or the OS-to-browser and OS SDK-to-in-app-webview mechanics, but it gets a little tricky...tricky in a fashion that bots of course can emulate as well. For the typical end user of a web browser using normal web browser settings just visiting websites and clicking things and getting the behavior the web browsers do, the headers are predictable; when they have extra plugins and other services layered on top of their browser that expressly change this behavior, it can get complicated, of course.

And, it is also the case that some app developers (mobile and desktop apps) can do clever things with interception and re-writing of headers and URLs, proxies, intermediate caches, and so forth.

But, anyway, it's good to have the extra signals in X-Analytics, which can be bundled with indicators like whether its deemed user traffic or other signals about the likelihood of the traffic being legitimately sourced traffic (user or bot, or agentic browser acting on user's behalf or otherwise, all the usual and emerging stuff), as it will help with hazarding better guesses on fluctuations of the counts and fluctuations of ratios of counts by dimension.

Thank you @dr0ptp4kt @Khantstop @DTotten-WMF and @Snwachukwu for the very productive meeting yesterday.

The summary of it is:

• wprov parameter is a viable solution to measure impact of interventions with non-search partners, assuming we are able to negotiate its inclusion in the intervention
• Must be used in conjunction with referrer when possible to exclude scrapers copy-pasting the urls with wprov parameter from our counts.
• How do we ensure that the wprov data imports reliably into our data lake for impact tracking?
  • We need to map which wprov code identifies which partner so we can distinguish different intervention
  • Partners need to use the correct parameter formatting so that we are able to parse the XAnalytics field on our end
  • We’d likely need other signals (e.g. sec-fetch) to operationalize this parameter for our measurement.

Thanks @Miriam et al for your review and detailed notes! It's helpful to know that wprov is a viable solution for measuring impact. Confirming the intention is that being able to measure impact would always be a pre-requisite for our IR2 interventions.

Some updates on the Target discussion.
TL;DR: we are landing around a target of 100M pageviews as marginal gain obtained via non-search interventions.

Rationale:

• We looked at an estimate of candidate non-search interventions for FY27, ranked them by likelihood that we will partner with each platform, and attached an estimate of pageview lift based on existing data. (yields a ~160M pageview target)
• We also looked at the ratio of search to non-search traffic (70:30) and worked out an estimate for IR2 targets by applying the ratio to IR1 targets. (yields a ~210M pageview target, an uppoer bound given that we don't know how much of the non-search traffic is actually third parties that don't send us a referrer)
• Given the amount of unknowns in this scenario, we are using a lower end estimate of a 100M PV/year gain (this is 0.29%, or one day, of all non-search traffic for the year) as the target.

Next steps:

• @RHo reviews the target estimates with @SCherukuwada
• Work out the KR targets for Q2 to be around 10-20% of the total (to account for the in flight or not yet implemented by partners).