Schema QA : mediawiki_reading_depth
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	jwang
	Dec 16 2021, 10:47 PM

Description

As one of the instrumentations for talk page, Web team has deployed the instrumentation to track read depth on talk page. T294777

Instrumentation note

The related events will be stored in a new schema mediawiki_reading_depth. Sample rate is 0.1% on English Wikipedia.

QA summary

What has been checked	Status	Note
Daily events	✅pass	The events are available sine 12-20-2021
Events by wiki projects	✅pass	Enabled only on english wikipedia
Events by namespace	✅pass	namespace id is captured
Events by user type	✅pass	Both anonymous user and logged in user are captured
Events by action	✅pass	Two types of actions are recorded: pageLoaded, pageUnloaded
Events by page_length	✅pass	Page length is recorded. Round down to the 1st digit
Events by access method	✅pass	Two types of access methods are recorded: desktop, mobile web
Events by agent type	✅pass	We can identify and exclude spider in analysis
Read length	✅pass	Reading time is captured. For all events in 2021, the average time length is 1178408, range [0, 2603264579]

Please see QA notebook for details.

Bugs/Potential Issues

Issues	Status	Note
`dt` field doesn't match with the partition year, month day	✅Not a bug	T292586#7581979

Related Objects
Search...

Status	Assigned	Task
Open	None	T158181 Aim for workflow equivalence for MediaWiki on desktop and mobile web
Open	None	T54165 Links to talk pages in mobile view for all anonymous users
Resolved	ovasileva	T293946 Enable talk for mobile users on enwiki
Open	None	T276497 Scale DiscussionTools to all projects
Open	None	T251207 [Epic] Scale DiscussionTools to all Wikipedias
Open	None	T313976 [Release Ticket] Offer complete set of DiscussionTools features to to all wikis on mobile
Open	ovasileva	T294503 Measuring the impact of exposing talk pages to mobile web anons
Resolved	jwang	T295915 Schema QA for measuring impact of exposing talk pages to mobile anons
Resolved	jwang	T297921 Schema QA : mediawiki_reading_depth

Event Timeline

jwang created this task.Dec 16 2021, 10:47 PM

@ovasileva , similar to T292586#7576004, The data in dt field doesn't match with the partition year, month day. It stored the date as early as 1986. It also stored future dates, in 2022, etc.

A few examples:

date_time	year	month	day	events	sessions
1986-08-22	2021	11	23	2	1
2021-12-14	2021	12	12	1	1
2021-12-14	2021	12	16	1	1
2021-12-15	2021	12	15	300833	71457
2021-12-16	2021	12	14	15	2
2021-12-18	2021	12	12	3	2
2021-12-18	2021	12	16	3	2
2021-12-19	2021	12	15	2	1
2021-12-24	2021	11	25	2	1
2022-01-02	2021	12	5	2	1

Query code

SELECT TO_DATE(dt),  year, month,day, COUNT(1) AS events, 
COUNT(DISTINCT session_token) AS sessions
FROM event.mediawiki_reading_depth
WHERE year=2021
GROUP BY TO_DATE(dt),year, month,day

jwang mentioned this in T295915: Schema QA for measuring impact of exposing talk pages to mobile anons.Dec 16 2021, 10:53 PM

jwang updated the task description. (Show Details)Dec 21 2021, 3:18 AM

jwang updated the task description. (Show Details)Dec 21 2021, 5:34 AM

Jdlrobson moved this task from Incoming to Analyst Consultation on the Web-Team-Backlog-Archived board.Dec 23 2021, 4:40 PM

@ovasileva I completed post-deployment QA of the instrumentation added to track read depth. All events appear to be recording as expected.

What has been checked	Status	Note
Daily events	✅pass	The events are available sine 12-20-2021
Events by wiki projects	✅pass	Enabled only on english wikipedia
Events by namespace	✅pass	namespace id is captured
Events by user type	✅pass	Both anonymous user and logged in user are captured
Events by action	✅pass	Two types of actions are recorded: pageLoaded, pageUnloaded
Events by page_length	✅pass	Page length is recorded. Round down to the 1st digit
Events by access method	✅pass	Two types of access methods are recorded: desktop, mobile web
Events by agent type	✅pass	We can identify and exclude spider in analysis
Read length	✅pass	Reading time is captured. For all events in 2021, the average time length is 1178408, range [0, 2603264579]

Please see QA notebook for details.

jwang updated the task description. (Show Details)Jan 5 2022, 6:17 AM

Perfect, thank you @jwang! Resolving this one

ovasileva mentioned this in T292587: Sticky header: Create A/B test schema and tie to sticky header feature.Jan 5 2022, 11:35 AM

Schema QA : mediawiki_reading_depthClosed, ResolvedPublicActions