Page MenuHomePhabricator

Analyze session length on English Wikipedia
Closed, ResolvedPublic

Description

Background

We would like to measure the session length for the Vector 2022 skin post deployment analysis on enwiki.

Acceptance criteria

  • Compare session length before, immediately after, and two/three weeks after the Vector 2022 deployment. Note any interesting shifts or patterns.

Results due: 22 Feb 2023

Event Timeline

jwang updated the task description. (Show Details)

Hi @ovasileva, created this ticket to track analysis of session length. Please add acceptance criteria for a clearer scope. Thanks!

jwang triaged this task as High priority.Feb 7 2023, 6:05 PM
MNeisler moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
MNeisler added a subscriber: ovasileva.

See report and summary of findings below. Let me know if additional details would be useful.

Overall, there appear to be no significant changes in session length immediately following or two weeks following the deployment of Vector 2022 on English Wikipedia.

Session length distribution

Session length percentiles on English Wikipedia.png (441×712 px, 19 KB)

We did not observe any changes in session length distribution as indicated by the daily session length percentile chart above. The percentiles have remained as follows two weeks prior and two weeks post deployment:

  • 25th percentiles: 0 minutes (Note: Session length = 0 includes sessions of length up to, but not including, 1 minute)
  • 50th percentile (median): 1 minute (Half of session lengths were under 1 minute)
  • 75th percentiles: 5 minutes
  • 95th percentile: 29 minutes

Average session length

Weighted average based on the distribution of session lengths

daily_avg_session_length_enwiki.png (2×4 px, 168 KB)

No significant changes. The average session length on English Wikipedia two weeks prior to deployment was 5.9 minutes and it was 5.8 minutes following deployment.

Here are the daily session length averages recorded in the immediate days prior and post deployment on 18 Jan 2022:

DateAverage session length (minutes)
2023-01-145.75
2023-01-155.83
2023-01-165.97
2023-01-176.03
2023-01-185.90
2023-01-195.85
2023-01-205.78
2023-01-215.68
2023-01-225.78

Number of Sessions

*Estimation of session count based on the 1:10 sampling rate.

daily_session_count_enwiki.png (2×4 px, 183 KB)

There were about 1.3B total sessions logged two weeks before and two weeks after deployment of Vector 2022.
Two weeks prior: 1,295,477,260 total sessions
Two weeks post: 1,285,976,140 total sessions

Here are the number of sessions logged in the immediate days following deployment:

DateSession count
2023-01-1484.9 M
2023-01-1591.0 M
2023-01-1689.7 M
2023-01-1786.9M
2023-01-1885.1 M
2023-01-1985.6M
2023-01-2084.1M
2023-01-2183.2M
2023-01-2289.9M

Notes:

  • Data comes from the wmf.session_length_daily table and limited analysis to session lengths on English Wikipedia.
  • Data is limited to all session lengths logged on English Wikipedia and reflects usage across all available desktop skins (The dataset used in the analysis did not track skin type).
  • Session length measures the time users engage with a site in increments of 1 minute, rounded down. Sessions end after 30 minutes of inactivity and exclude bots that crawl Wikipedia sites using javascript. More details can be found on the wikitech session length page.
  • There is an existing session length dashboard that can be used to monitor many of these trends on an ongoing basis.

Let me
cc @ovasileva @jwang