Yearly site performance reports (blog post)
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Peter
	Aug 24 2015, 10:51 PM

Description

We should be able to publicly show the performance improvement/regressions in a good format to the rest of the world. It's a good way for us to share how we are doing and what we actually do. The reports need to hold in detail information but also not be too techie.

Check out Etsy's great performance reports for inspiration
https://www.etsy.com/uk/codeascraft/q1-2016-site-performance-report
https://codeascraft.com/2014/12/22/q3-2014-site-performance-report/

High level of what todo:

Define what to include
Decide the format
First release date the end of Q3

See:

T181959: Blog post: Why we measure performance

Related Objects

Mentioned Here: T187684: Define key metrics (3-5 total) that we're tracking and go public with the metrics
T181959: Blog post: Why we measure performance
T110121: Monitoring: What interactions should we monitor?

Event Timeline

Peter created this task.Aug 24 2015, 10:51 PM

Peter raised the priority of this task from to Medium.

Peter updated the task description. (Show Details)

Peter added a project: Performance-Team.

Peter subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 24 2015, 10:51 PM

The Etsy reports are really good, I think following their pattern with a few changes will be great as a start. What about something like this: Long time goal should be to report median and 95th percentile.

Synthetic Front-End Performance - reporting SpeedIndex per browser and anonymous/logged in user and the baseline page. Should we also use other numbers so we are not forced to use WebPageTest?
Real User Front-end Performance - TBD
Server Side Performance - TBD
Extras: At first we report save performance and then add extra stats that will come out of T110121

ori moved this task from Inbox, needs triage to To-do: Goals prioritized current Quarter on the Performance-Team board.Sep 14 2015, 5:54 PM

Krinkle subscribed.Dec 6 2016, 12:15 AM

• Gilles renamed this task from Quarterly site performance reports to Yearly site performance reports.Dec 7 2016, 5:34 PM

• Gilles raised the priority of this task from Medium to High.

• Gilles set Security to None.

• Gilles moved this task from To-do: Goals prioritized current Quarter to Backlog: Maintenance, non-prioritized on the Performance-Team board.

Krinkle renamed this task from Yearly site performance reports to Quarterly performance reports (blog post).Mar 23 2017, 8:57 PM

Krinkle moved this task from Backlog: Maintenance, non-prioritized to Inbox, needs triage on the Performance-Team board.

Change 344504 had a related patch set uploaded (by Krinkle):
[operations/mediawiki-config@master] StartProfiler: Disable production sampling for xhgui

https://gerrit.wikimedia.org/r/344504

gerritbot added a project: Patch-For-Review.Mar 23 2017, 9:29 PM

Krinkle removed a project: Patch-For-Review.Mar 23 2017, 9:31 PM

• Gilles moved this task from Inbox, needs triage to Backlog: Maintenance, non-prioritized on the Performance-Team board.Mar 29 2017, 7:32 PM

Krinkle renamed this task from Quarterly performance reports (blog post) to Yearly site performance reports (blog post).Jun 14 2017, 7:31 PM

Krinkle moved this task from Backlog: Maintenance, non-prioritized to To-do: Goals, prioritized next 4 Quarters on the Performance-Team board.

We mentioned this in the meeting last week. We're still interested in doing these, but we do need better stability in our metrics to be able to do this in a meaningful way, and to be strict about investigating regressions and improvements as they happen.

We'e been doing this for about 2-3 months, largely thanks to our new alerts. Reprioritising this for now, but we should consider this 1-2 quarters from now.

• Imarlier subscribed.Dec 4 2017, 9:55 PM

Krinkle updated the task description. (Show Details)Jan 18 2018, 3:45 PM

• Imarlier claimed this task.Jan 18 2018, 5:28 PM

• Imarlier moved this task from To-do: Goals, prioritized next 4 Quarters to To-do: Goals prioritized current Quarter on the Performance-Team board.Apr 30 2018, 7:45 PM

Ref: the post that Gilles wrote last year.

It doesn't make sense to work on this until we've completed the Performance Perception survey and analysis -- that work will help us to identify a small number of metrics that we should be focusing on (and reporting on) rather than trying to address everything at once, or naively assuming that certain metrics are important without knowing for sure.

• kchapman removed • Imarlier as the assignee of this task.Jan 25 2019, 2:49 AM

Done, per https://phabricator.wikimedia.org/phame/post/view/124/perf_matters_at_wikipedia_2015/.

For now, it covers individual projects and metric impacts, as opposed to the a regular reporting of key metrics. In part because T187684 is not yet resolved, but also because we lost the Graphite data for 2015/2016. I'm hoping after T187684 is resolved, starting with the 2017 edition we'll do year-range analysis of metrics as well.

Krinkle closed this task as Resolved.Feb 19 2019, 10:01 PM

Krinkle moved this task from Blocked (old) to Doing (old) on the Performance-Team board.

Krinkle updated the task description. (Show Details)Nov 30 2022, 2:02 PM

Yearly site performance reports (blog post)Closed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Yearly site performance reports (blog post)
Closed, ResolvedPublic
Actions