Gather information about devices, network speeds to run tests about
Closed, DeclinedPublic
Actions

Assigned To

Authored By

	• Jhernandez
	Oct 17 2017, 4:45 PM

Description

In order to define proper tests and comparisons we need specific information about the types of devices and networks we expect from users.

Devices

Via https://commons.wikimedia.org/wiki/Category:New_Readers_user_personas
- XOLO Q700s
- iPhone 6
- Blackberry Z10
- Infinix Android Race Eagle X50
- Forme Discovery P9
- LeEco Le 1s
- Xolo Era
- Tecno L8
Via Performance Budgets:
- Moto G4 or Moto G5 Plus
Via https://meta.wikimedia.org/w/index.php?title=File:Movement_Strategy_-_New_Voices_Research_-_Findings_%26_Opportunities_from_Indonesia_%26_Brazil.pdf&page=2
- Motorola Moto X
- OPPO N1
- Moto G3
- Samsung Galaxy S5
- LG Optimus L3 2
- Samsung Grand Duo
- Samsung Galaxy S4
- Samsung Galaxy S3

Connection speeds, bundle sizes, time-to-interactivity, cache lifespan

TBD

Via Performance Budgets:
- 400ms RTT, 400Kbps
- 130KiB total compressed data for critical path JS, CSS, client HTML / data request, and SSR HTML on the Barack Obama article endpoint (wiki route)?*
- 5s cold load TTI
- 2s warm load TTI
What's our budget for the number of critical path HTTP requests? Batching requests may be opposed to cache lifespan.
Since our content abstraction is so slim, do we want to budget or measure client requests? e.g., https://en.wikipedia.org/api/rest_v1/page/mobile-sections/Carrot_cake.

*This budget seems to imply only presenting a small portion of article content, such as a section, for SSRs.

References

Performance Budgets
Current mobile and desktop performance

Details

	Subject	Repo	Branch	Lines +/-
	Chore: add Webpack performance requirements	marvin	master	+10 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Declined		• Jhernandez	T177058 Set up performance tooling & dashboards
		Declined		ovasileva	T178403 Gather information about devices, network speeds to run tests about

Event Timeline

It may be helpful to consider the personas and information from the new readers reports, and other profiles from the developed countries to have a comprehensive list.

• Jhernandez removed • Jhernandez as the assignee of this task.Oct 17 2017, 4:46 PM

here's a list of the old devices from https://commons.wikimedia.org/wiki/Category:New_Readers_user_personas:

XOLO Q700s
iPhone 6
Blackberry Z10
Infinix Android Race Eagle X50
Forme Discovery P9
LeEco Le 1s
Xolo Era
Tecno L8

@ovasileva (thanks!), @Jhernandez, should we focus on a subset (newer is probably better, one OS instead of many could help too) so that if we go to production early our compatibility problems will be limited?

• Niedzielski moved this task from Needs Triage to Backlog (Priority Descending) on the Marvin board.Oct 31 2017, 7:13 PM

This https://infrequently.org/2017/10/can-you-afford-it-real-world-web-performance-budgets/ has some interesting data, and it mentions the Moto G4 for last year and the Moto G5 for this year:

The true median device from 2016 sold at about ~$200 unlocked. This year’s median device is even cheaper, but their performance is roughly equivalent. Expect continued performance stasis at the median for the next few years. This is part of the reason I suggested the Moto G4 last year and recommend it or the Moto G5 Plus this year.

Putting it all together, our global baseline for performance benchmarking is a:

~$200 (new, unlocked) Android phone
On a slow 3G network, emulated at:
400ms RTT
400Kbps transfer