Page MenuHomePhabricator
Create Task
Maniphest T105681

Create media performance dashboard
Closed, ResolvedPublic
Actions

Description

Images make up the bulk of the page weight, they should have their own dedicated grafana dashboard.

The first thing to track should be thumbnail generation time, in order to study the effect of T105680 beind deployed.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Send image varnish frontend data from logs to statsdoperations/puppetproduction+255 -73
Backport Media Viewer image load time statsv trackingmediawiki/corewmf/1.26wmf16+1 -1
Add link to media dashboardperformance/docrootmaster+1 -0
Add most common non-bucket size to statsv performance trackingmediawiki/extensions/MultimediaViewermaster+4 -1
Track image load time with statsvmediawiki/extensions/MultimediaViewerwmf/1.26wmf16+10 -2
Track image load time with statsvmediawiki/extensions/MultimediaViewermaster+8 -0
Refine thumbnailing metricsmediawiki/coremaster+29 -8
Record timing of thumbnail generation and store pullmediawiki/coremaster+11 -0
Customize query in gerrit

Related Objects

Event Timeline

• Gilles created this task.Jul 13 2015, 10:06 AM
• Gilles claimed this task.
• Gilles raised the priority of this task from to Medium.
• Gilles updated the task description. (Show Details)
• Gilles added a project: Performance-Team.
• Gilles added subscribers: Aklapper, • Gilles.
gerritbot subscribed.Jul 14 2015, 11:32 AM

Change 224599 had a related patch set uploaded (by Gilles):
Record timing of thumbnail generation and store pull

https://gerrit.wikimedia.org/r/224599

gerritbot added a project: Patch-For-Review.Jul 14 2015, 11:32 AM
• Gilles moved this task from Inbox, needs triage to Doing (old) on the Performance-Team board.Jul 14 2015, 12:37 PM
gerritbot added a comment.Jul 15 2015, 5:28 AM

Change 224599 merged by jenkins-bot:
Record timing of thumbnail generation and store pull

https://gerrit.wikimedia.org/r/224599

• Gilles mentioned this in rMW103eff5bcdaf: Record timing of thumbnail generation and store pull.Jul 15 2015, 5:28 AM
• Forrestbot added projects: MW-1.26-release, WMF-deploy-2015-07-21_(1.26wmf15).Jul 15 2015, 6:00 AM
gerritbot added a comment.Jul 17 2015, 11:49 AM

Change 225304 had a related patch set uploaded (by Gilles):
Refine thumbnailing metrics

https://gerrit.wikimedia.org/r/225304

gerritbot added a comment.Jul 17 2015, 7:03 PM

Change 225304 merged by jenkins-bot:
Refine thumbnailing metrics

https://gerrit.wikimedia.org/r/225304

• Gilles mentioned this in rMWec60612b6f5f: Refine thumbnailing metrics.Jul 17 2015, 7:03 PM
• Gilles mentioned this in T100976: Thumbnails should embed tinyRGB instead of sRGB when they include that ICC profile.Jul 20 2015, 11:14 AM
• Gilles removed a project: Patch-For-Review.Jul 20 2015, 11:24 AM
• Gilles set Security to None.
• Gilles added a comment.Jul 20 2015, 11:11 PM

https://grafana.wikimedia.org/#/dashboard/db/media

• Gilles added a comment.Jul 21 2015, 8:25 PM

varnishlog command on deployment-cache-upload02 to get the data that matters:

 varnishlog -m RxURL:/thumb/ -n frontend -i Hit -i HitPass -i RxURL -i ReqEnd -i TxStatus

13 RxURL        c /wikipedia/de/thumb/2/24/Wikinews-logo.svg/26px-Wikinews-logo.svg.png
13 TxStatus     c 304
13 ReqEnd       c 1770922296 1437510136.149027109 1437510136.149499655 0.129729748 0.000418901 0.000053644

The float before last is the one that matters most: it's how long the backend took to produce the file.

gerritbot added a comment.Jul 23 2015, 4:48 PM

Change 226556 had a related patch set uploaded (by Gilles):
Track image load time with statsv

https://gerrit.wikimedia.org/r/226556

gerritbot added a project: Patch-For-Review.Jul 23 2015, 4:48 PM
gerritbot added a comment.Jul 31 2015, 8:35 AM

Change 226556 merged by jenkins-bot:
Track image load time with statsv

https://gerrit.wikimedia.org/r/226556

• Gilles mentioned this in rEMMV66d510dddc69: Track image load time with statsv.Jul 31 2015, 8:35 AM
Diffusion mentioned this in rMEXTae2f9fc00f1f: Updated mediawiki/extensions Project: mediawiki/extensions/MultimediaViewer….Jul 31 2015, 8:35 AM
gerritbot added a comment.Jul 31 2015, 8:49 AM

Change 228218 had a related patch set uploaded (by Gilles):
Track image load time with statsv

https://gerrit.wikimedia.org/r/228218

gerritbot added a comment.Jul 31 2015, 8:56 AM

Change 228218 merged by jenkins-bot:
Track image load time with statsv

https://gerrit.wikimedia.org/r/228218

• Gilles mentioned this in rEMMV6930738be85a: Track image load time with statsv.Jul 31 2015, 8:56 AM
• Gilles mentioned this in rMWf062eeab53c3: Updated mediawiki/core Project: mediawiki/extensions/MultimediaViewer….
• Forrestbot added projects: WMF-deploy-2015-08-04_(1.26wmf17), WMF-deploy-2015-07-28_(1.26wmf16).Jul 31 2015, 9:00 AM
gerritbot added a comment.Jul 31 2015, 9:02 AM

Change 228219 had a related patch set uploaded (by Gilles):
Backport Media Viewer image load time statsv tracking

https://gerrit.wikimedia.org/r/228219

• Gilles removed a project: Patch-For-Review.Aug 5 2015, 8:00 AM
• Gilles renamed this task from Create image performance dashboard to Create media performance dashboard.Aug 5 2015, 8:27 AM
gerritbot added a comment.Aug 5 2015, 9:01 AM

Change 229340 had a related patch set uploaded (by Gilles):
Add most common non-bucket size to statsv performance tracking

https://gerrit.wikimedia.org/r/229340

gerritbot added a project: Patch-For-Review.Aug 5 2015, 9:01 AM
gerritbot added a comment.Aug 5 2015, 9:18 AM

Change 229341 had a related patch set uploaded (by Gilles):
Add link to media dashboard

https://gerrit.wikimedia.org/r/229341

gerritbot added a comment.Aug 5 2015, 6:35 PM

Change 229340 merged by jenkins-bot:
Add most common non-bucket size to statsv performance tracking

https://gerrit.wikimedia.org/r/229340

• Gilles mentioned this in rMEXT2cda4c822118: Updated mediawiki/extensions Project: mediawiki/extensions/MultimediaViewer….Aug 5 2015, 6:35 PM
• Gilles mentioned this in rEMMV04be1d5fcc73: Add most common non-bucket size to statsv performance tracking.
• Forrestbot added a project: WMF-deploy-2015-08-11_(1.26wmf18).Aug 5 2015, 7:00 PM
Krinkle removed projects: WMF-deploy-2015-08-11_(1.26wmf18), WMF-deploy-2015-07-28_(1.26wmf16), WMF-deploy-2015-08-04_(1.26wmf17), WMF-deploy-2015-07-21_(1.26wmf15), MW-1.26-release.Aug 5 2015, 7:48 PM
gerritbot added a comment.Aug 5 2015, 7:51 PM

Change 229341 merged by jenkins-bot:
Add link to media dashboard

https://gerrit.wikimedia.org/r/229341

• Gilles removed a project: Patch-For-Review.Aug 26 2015, 7:08 AM
gerritbot added a comment.Aug 26 2015, 9:18 PM

Change 228219 abandoned by Gilles:
Backport Media Viewer image load time statsv tracking

https://gerrit.wikimedia.org/r/228219

gerritbot added a comment.Aug 26 2015, 9:46 PM

Change 234157 had a related patch set uploaded (by Gilles):
Send image varnish frontend data from logs to statsd

https://gerrit.wikimedia.org/r/234157

gerritbot added a project: Patch-For-Review.Aug 26 2015, 9:46 PM
• Gilles moved this task from Doing (old) to Backlog: Maintenance, non-prioritized on the Performance-Team board.Sep 10 2015, 7:34 PM
gerritbot added a comment.Sep 14 2015, 5:18 PM

Change 234157 merged by Ori.livneh:
Send image varnish frontend data from logs to statsd

https://gerrit.wikimedia.org/r/234157

ori mentioned this in rOPUPcd6796269374: Send image varnish frontend data from logs to statsd.Sep 14 2015, 5:18 PM
• Gilles moved this task from Backlog: Maintenance, non-prioritized to Doing (old) on the Performance-Team board.Sep 14 2015, 6:05 PM
• Gilles closed this task as Resolved.Sep 15 2015, 2:32 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits