whats the priority on this?

Hi DPE team, Can you pls let me know the status of this request? I was not able to get any results for Sec-Purpose: Prefetch; anonymous-client-ip when querying the x_analytics field.
And a follow up question, can we get this added as a flag or new column to the readers tables derived from webrequest - pageviews_hourly and pageview_daily so its easier for us to query and visualize? pls let me know if this requires a new phab task.

The Data Stewardship RASCI: Roles and responsibilities is available here. It was reviewed by the working group and we have their buy-in.
We also completed onboarding the Contributors metrics data stewards. Starting January 2024 we will be training the data stewards.

We have discovered an issue with unique devices which is reported and being fixed here T353296

Ok found it on email : SDS 1.3 Essential Metrics Decision Making Proposal v2

In T342472#9393767, @YLiou_WMF wrote:

• Revised draft proposal shared with full core and extended group and Leila on December 8.

Article is published here- Announcing Wikipedia’s most popular articles of 2023

Blocks have increased on all our major wikis starting October end. We are investigating to see if some of those blocks are impacting human editors more than we would want them to.

• The surge of blocks on single IP and IP range on eswiki is mainly due to the activity of BlockBot-es. In history, there was a similar bump between November 2021 and August 2022, which peaked in December 2021 (see the IP masking dashboard)

I'm waiting on Oct 2023 content metrics to be updated before I upload the slides to Commons

Adding this Comparison sheet of the tables that provide Editors by Geo that was created by @Iflorez
In this sheet you can see the differences in actual active editors by region in the tables we use that are all considering the same definition of active editor.

thanks @Anoop ! do you have data on any increase in content added to kn wikipedia during that time? so we know it corresponds with the rise in pageviews.
I dont see any significant increase in translations from our internal data.

thank you @mpopov !! <3

thanks @fkaelin !
@Hghani is working on replacing the csv files with Hive table data and hopefully we will use that from next snapshot onwards.

@fkaelin can you pls ping us here, or on Slack to indicate when the content metrics csv data is available with September 2023 snapshot? we are expecting to have it on 10/23 or 10/24. pls let us know if there are any changes to this expectation. thanks!

Ok lets wait until 10/24 to calculate and publish monthly Content metrics.

Completed and Signing off on the QA for monthly and quarterly Relevance and Content metrics.

@OSefu-WMF , this would be the first quarter where we're reporting on these metrics. Ive pulled baselines for Content and Relevance before.
Content: can be calculated using the content notebooks here - https://github.com/wikimedia-research/core-annual-plan-metrics/tree/main
Relevance: https://github.com/wikimedia-research/movement-metrics/blob/master/metrics/regional_unique_devices.tsv

@nshahquinn-wmf : wanted to make sure you were aware of the decision for Gender metrics T346160#9211877 .
when I calculated the baseline I only used male, female and non-binary categories but it should be modified as mentioned in the decision record above.

@CMyrick-WMF do you mind opening a new task for the Q1 data? since the original request was completed within the due date.

This is completed.

Assigned to @OSefu-WMF for running the notebooks and creating the Visualizations.

Thanks for confirming @kai.nissen !
I checked our dashboards and can see a drop in the unique devices from Germany https://superset.wikimedia.org/superset/explore/p/DJVXakAnYMw/
@MGerlach : I see a rise in unique devices in India. corresponds with a banner that ran in India between 9/1 and 9/20 https://meta.wikimedia.org/wiki/Special:CentralNotice

I created a Github Repo: https://github.com/wikimedia-research/core-annual-plan-metrics/tree/main
and added the Content metric notebooks that I used to generate the baseline metrics.

This feels like an effect of Chrome's UA reduction where in Phase 5, the device OS was replaced. See Rollout details here

@CorinnaHillebrand_WMDE , can you please confirm if this is completed? and when was it deployed?

Oops! I forgot to add here but had sent a private Slack message to @kzimmerman few weeks ago -
added the June snapshot of quality articles for Gender Gap in this spreadsheet.
the last time we tried to calculate the latest snapshot for gender data we ran into an issue where the numbers were very low T343289. the issue seems to be resolved now, looks like there was something going on with the csv that I was using..the June snapshot I recalculated is more in line with the previous months.

Hi @SNowick_WMF : do you think this task is still relevant? do the counts differ between legacy and MEP for mobile apps? can you pls decline this task if thats not the case. and if it is, then reassign to @nettrom_WMF ?

in light of the discussions in T288983#9046285 . I'm declining this task.

In Hua's repo:
For map: https://github.com/wikimedia-research/key_product_metrics/blob/main/wikicharts/map.ipynb, see Quality Articles map (monthly, proportion, YoY change)
For time series, we have active editors and unique devices

• note that we would not want to change any of our existing dimensions (like agent_type) to indicate prefetch pageviews since this will break our reporting and has consequences on Superset dashboards. Instead find a way to store this in any existing field or create new field

In T336715#9132510, @Mayakp.wiki wrote:

Action Items:

• What happens when a user clicks on the pre fetched page? Is that recorded as a pageview?
  • Connect with partnerships to do factfinding with Google
  • Dig into our data on user traffic and how we’re segmenting traffic
• Circle with DE and discuss about this

Relevant materials:
relevance: Unique Devices by Region
Content by geo: Evolution of regional content
Content by gender: Knowledge Gaps Plots - Queering Wikipedia

Added Movement-Insights team tag to accurately reflect current work. Keeping a close tab on T344919 as this task will inform the impact of IP Masking on datasets and pipelines.

There are several questions that would be great to get more answers from Partnerships and Google:

• they should have statistics on the number of pre-fetches to our site (since it goes through their private proxy). does this match our estimated numbers?
• clarify how they make requests for pages that users navigate to but are already pre-fetched
• statistics on the speed benefit for users. how large is the benefit of this feature for users (ideally depending on the region)? there is a cost-benefit trade-off (for us). if we had pre-fetched everything for everyone, the time to serve would be very small; however, our infrastructure couldnt cope with this. so how large is the improvement in user-experience compared to the additional cost of serving 2B pageviews per month? for example, if there is 1ms improvment per 1B additional pageviews it would not be worth it. but maybe if it was 100ms per 1B additional pageviews? I could imagine folks from SRE/Traffic have opinions/expertise on this; I am sure they have some metrics/dashboards on latency but I cant find this now. this blogpost mentions some metrics around this (so maybe Brandon might be someone to reach out to about this?)

Posted on Slack and Commons

Decisions:
With the assumption that we tag pre fetch requests as automated traffic and don't see a corresponding decline in User traffic, we don't see an immediate need to turn off this feature.

• another reason to continue this feature is because it enhances user experience by reducing load times, which is beneficial for the consumers of our projects and could lead to increase in unique devices (which is an annual plan metric that we are aiming to move up).

@mpopov This is completed. @nshahquinn-wmf and I worked out a plan to incorporate the viz into our monthly reporting.

We are meeting this week to make a decision to turn off pre-fetch from Wikimedia-side (T218618#9028512 ) to potentially reduce 1-2B requests per month.

cc: @odimitrijevic , @Milimetric
tagging Data-platform-engineering as FYI. Once this is fixed by the WMDE Fundraising team we would expect to see a drop in the unique devices to wikisource from Germany (unique_devices_per_project_family_monthly table).

I tried a few things -

1. created a new notebook today for calculating the values using the gender content gap csv. and I'm now getting values similar to yours, though slightly different than yours.

gender3category article_created_value
0 female 2381837
1 male 9873151
2 non-binary 13463

Hi @tmletzko, Can you please help us with an issue we're noticing which is possibly due to how CentralNotice behaves?
We observed inflated numbers for unique devices in smaller projects like Wikisource during particular months of the year and the reason for that seems to be that the closing of the banner on German Wikipedia seems to trigger a 302-request to some of the other projects (like wikisource etc.) . we were hoping to validate if this issue is indeed happening due to CentralNotice and then open a new task for your team to fix, if need be.