@ovasileva, please find the raw data of above graph in google sheet.

@mforns @BTullis , thank you both for explaining the details. I consider this issue is fix and mark it as resolve.

@BTullis, thank you very much for working on this issue. I tried the query . It failed with a different error. (please see log below).
Log of error:

---------------------------------------------------------------------------
ChildProcessError                         Traceback (most recent call last)
<ipython-input-29-014bde41aa40> in <module>
----> 1 df_edits_quality_damaging_rate_nonbot_registered_user=hive.run(query_edits_quality_damaging_model_nonbot_registered_user.format(YEAR=yr, START_YYYY_MM_DD=start_date_YYYY_MM_DD , END_YYYY_MM_DD=end_date_YYYY_MM_DD))

• The pageviews of talk pages from mobile web logged-out users increased significantly since Nov 15, 2021
• The pageviews of talk pages from mobile web logged-out users has started to grow before the official enable date. It started to grow on Nov 8, 2021, and spiked between Nov 09, 2021 and Nov 11,2021 for the first time. We are not aware of the reason which caused the spike before Nov 15, 2021.
• The pageviews of talk pages from mobile web logged-out users is not 0 before the official enable date. It might be due to mis-categorization of access method.
• Based on the above observation, we select 2 weeks between 2021-10-24 and 2021-11-06 as the pre-deployment stage. We selected 2 weeks between 2021-11-28 and 2021-12-11 as the post-deployment stage. We estimated that the daily pageviews grew 195k due to enabling talk page to mobile web logged-out users.

Below graph shows the bounce rate on talk page for all users after talk page was enable on Nov 15, 2021. By the end of Dec 2021, the bounce rate varied between 54.35% and 60.68%, with an average of 57.91%. The average bounce rate of logged-in users is 37.00% between Nov 15, 2021 and Dec 31, 2021. If we assume the bounce rate of logged-in users keep same pre and post the change, we estimate that bounce rate increased 20.91% due to enabling talk page for mobile web anonymous users.

Note: we defined reverts as the number of edits which were reverted within 48 hours after editing

@Jdlrobson , @ovasileva I have created an instrumentation spec and documented the events that need to be tracked to calculate each metric.

@kzimmerman , in theory, blocks will directly impact edits top-level metric. It might not be noticeable if blocks are only applied on small portion of total users.
It possibly impacts new account creation from the blocked IP. But user still can create account by emailing admin (ref: Global block FAQ)
It is mentioned on Global IP block that "Global blocks will remove talk page access when the block is encountered". If the access means pageview access, then it might impact the pageviews on talk page. I checked trend on enwiki from en6C. Pageviews on talk page is not declining 21/22 Q1 and Q2. ( superset link )

@Huji, the graph is correct by its definition. The retention rate definition: out of the non-bot users who registered in the week before the previous and made at least one edit in their first 30 days, the proportion who also edited during their second 30 days. The graph shows the retention rate of each user cohort. As of now (2021-01-06), we are only able to calculate the retention rate of the users who created account in week 44. The number beyond 44th week is an underestimation of retention rate because the recent cohorts of users are still in 30+30 days editing window.

Persian (Farsi) Wikipedia community started blocking IP editing on content pages since October 20,2021, which is the Wednesday of 42th week in 2021. In the below week over week comparison, to minimize seasonality impact, we choose the 49th week (2021-12-06 ~ 2021-12-12) , which does not include any holidays. We do not compare the trend year over year, because too many factors are impacting user behavior. One of the factors is pandemic, which impacted user behavior in the last two years and is still having the impact.
Here are observations 8 weeks after the change, i.e. 49th Week of 2021 compared to 41st week. After turning off IP editing on article page on fawiki, we saw:

• No significant increase in active registered editors, 1735 active registered editors in week 41, 1768 in week 49.
• No significant increase in new accounts, from 1778 in week 41 to 1785 in week 49. Turning off IP editing on article pages did not lead to a significant increase in new accounts.
• A 27% decrease in total edits, from 55753 in week 41 to 40505 in week 49
• A 45% decrease in reverts, from 5555 in week 41 to 3031 in week 49. The decrease is mainly from reverts on article content pages.
• A 18% decrease in net non-reverted edits excluding bot edits, from 35796 in week 41 to 29085 in week 49
• A 51% decrease in blocks From 543 in week 41 to 268 in week 49
• No obvious changes in Retention rate, Need more time to observe.
• No obvious trend on checkusers
• A 72% decrease in protected pages, from 157 in week 39 (week 41 is an abnormal week) to 44 in week 49. The decrease is mainly from protection on article content pages.

Please see 2021 report for details.

@ovasileva I completed post-deployment QA of the instrumentation added to track read depth. All events appear to be recording as expected.

@cjming , thanks for the information.

@ovasileva I completed post-deployment QA of the instrumentation added to track AB test enrollment. All events appear to be recording as expected.

@ovasileva I completed post-deployment QA of the instrumentation added to track scroll. All events appear to be recording as expected.

@kzimmerman , we have trend of blocks on IP masking dashboard. On enwiki, it did surged since August. Meanwhile on eswiki, blocks surged since November.

@ovasileva , do we have instrumentation to track clicks on sticky header? (instrumentation spec)

@nshahquinn-wmf, thank you very much for the info! Then it's not a bug.

@Jdlrobson. Understand. Thanks for the explanation.

@ovasileva , similar to T292586#7576004, The data in dt field doesn't match with the partition year, month day. It stored the date as early as 1986. It also stored future dates, in 2022, etc.

The latest data up to 2021-11-31 is available in Edit_topics_dashboard

Potential issue:
The data in dt field is not right. It doesn't match with the partition year, month day. Details in T292586#7576004.

Hi, I explored the mediawiki_web_ui_scroll schema. It seems that the data in dt field is not right. It records some dates in 2008 when this schema is not enabled. It also records many dates in 2022, which is the future. It doesn't match with the partition year, month day.
This data issue shows for all wikis not just on particular one wiki. Is it a known issue?

In general it depends. This request_id is used for request tracing. For example, if this event were generated by MediaWiki server side, it would be the X-Request-ID header set by MediaWiki. EventGate will create a new one for this if it isn't already set.
Basically, if all systems respect this and forward it properly, it is possible to reconstruct a 'what triggers what' series of requests and events in logs.

Hi @cjming, thanks for the info. Then what is the request referring to in the common fragment?

Explored data and found 2 possible issues.

• unsampled group is recorded in the data.
  • Status: engineer will fix it.

Explored data, have questions about the definitions of 2 fields, discussed at T292586#7574309 thread.

Hi, I find that 2 data fields in mediawiki_web_ui_scroll schema are not in original sticky header instrumentation spec. Can anyone help me to understand what they are tracking?

@Jdlrobson, My original understanding is that each token (session_id) will only be recorded once in this AB test schema. May I know why the token is repeatedly recorded in the schema? If one toke (session_id) is only sent into one experiment group, then the other rows of data are redundant.

3. Number of pages protected

The events are available in schema mediawiki_web_ab_test_enrollment now. But we only see control and unsampled in group field. We did not see treatment group yet.

@Huji Thanks for your questions. I created 2 sub tickets to track the followup work: T297655 , T297653. Some schemas are snapshot on a monthly basis. The metrics extracted from those schemas have one month lag. Retention rate has two months lag due to the nature of the metric definition.

Queried at 2pm on Dec 13,2021. No events were found in schema mediawiki_web_ab_test_enrollment

Tested the availability of the events for below scenarios on anonymous users:
✅ init events are available in schema

query condition:  event.action='init' and event.isanon=true

✅ events of clicking on article or talk page tab are available in schema

query condition: event.action='click'  and event.isanon=true and event.name='tabs.subject' for article tab or event.name='tabs.talk' for talk tab

✅ events of clicking "Add new discussion" are available in schema

query condition: event.action='click'  and event.isanon=true and event.name='talkpage.add-topic'

✅ events of clicking a section heading are available in schema

query condition: event.action='click'  and event.isanon=true and event.name='talkpage.section'

✅ events of clicking read as wiki page are available in schema

query condition: event.action='click'  and event.isanon=true and event.name='talkpage.readAsWiki'

discussed this in standup today. Does it make sense to do a quick check for this task and then document the full QA in T295915: Schema QA for measuring impact of exposing talk pages to mobile anons?

@Niharika, counting unique users is to answer below measurement questions (also mentioned in T292803). If below reframed measurement questions are OK to you, a unique token is enough for us.

Hi @mforns, could you review the patch?

The dashboard of fawiki metrics has now been set up. It will be refreshed every week and published report at https://analytics.wikimedia.org/published/notebooks/AHT/fawiki_dashboard.html .

@jwang Should I also update the ULS instrumentation to set a more helpful context like "sticky-header`?

@Edtadros, are the schemas ready for QA now? If so, could you confirm the schema names enabled by T292586 and T292587? I searched by key words "scroll" or "ab_test", etc.. I do not see the new schemas in event database yet.

Hi @Edtadros , is the schema named as web_ab_test_enrollment? If so, I do not see it is available in event database.