Page MenuHomePhabricator

Spike (8 hours): Demonstrate performance impact over the past quarter to mobile
Closed, ResolvedPublic

Description

We need to show performance improvements over the last quarter to report externally. Work with @JKatzWMF to make sure this gets nailed down.

Things we are looking for:

  • A few examples of how first paint was (where available), and the

status quo (easy to show)

  • Gather graphs showing first paint dropping over the quarter and document what has happened, where drops occur try and attribute them to specific events.
  • Work with @JKatzWMF to dig around more with the impact on page views (if any) given that people can now access our content quicker than before.
  • Investigate impact on EventLogging tables e.g. number of edits etc

Spike: 8hr

Event Timeline

Jdlrobson raised the priority of this task from to High.
Jdlrobson updated the task description. (Show Details)
Jdlrobson added subscribers: Jdlrobson, JKatzWMF.
KLans_WMF renamed this task from Spike: Demonstrate performance impact over the past quarter to mobile to Spike (8 hours): Demonstrate performance impact over the past quarter to mobile.Sep 14 2015, 4:37 PM
KLans_WMF set Security to None.
Jdlrobson moved this task from To Do to Doing on the Reading-Web-Sprint-56-Four Lions board.

I'm guessing this is on me. Have reached out to @JKatzWMF

I've dumped a bunch of information on the other Jon. Waiting on him now if he needs anything more.
The brief summary is:

  • Most of the notable work with stylesheet content in the head we did was done in May/June and the async change in July we reaped the benefits.
  • The graphs on http://grafana.wikimedia.org/#/dashboard/db/readingweb are our most useful tool for showing how the mobile site has improved over quarter.
  • First paint is lower on mobile than desktop on the same connection:
    • Special:Blankpage [1,2]: 1.681s vs 1.872s
    • Facebook [3,4]: 2.673s vs 4.384s
    • Barack Obama [5,6] 3.165s vs 4.283s
  • Problematically we have little historical data to accurately measure how impactful our changes have been
  • We did little tangible towards performance this quarter. The real successes were around uncovering big problems with the mobile site and forging plans to address them during the next 2 quarters (image and HTML size).
  • By giving attention to performance we've been able to notice regressions (for example the changes to Echo on desktop impacted our first paint). Getting to a point where we can detect regressions is an important next step.

[1] http://www.webpagetest.org/result/150928_0Z_1BV3/
[2] http://www.webpagetest.org/result/150928_EJ_1BTW/
[3] http://www.webpagetest.org/result/150928_Q5_1BTF/
[4] http://www.webpagetest.org/result/150928_B5_1BTE/
[5] http://www.webpagetest.org/result/150928_YY_1C09/
[6] http://www.webpagetest.org/result/150928_1V_1BZF/

@JKatzWMF can you sign this off or reopen if you need anything more? Thank you!