Fix commit title
exiv2 seems to fare better and find that the image isn't rotated. Unfortunately exiv2's support of ICC profile is new (latest version, 0.26) and only lets us extract the profile, not check its name first. It's going to be difficult to use it as a drop-in replacement for exiftool right now. Thumbor uses pyexiv2 (wrapper for exiv2), which means we might be able to at least read the orientation with pyexiv2, and keep doing the rest with exiftool for now, until exiv2 supports all the features we need.
Stock Mediawiki, OS X and Chrome all seem to agree that this image isn't rotated. But exiftool, which Thumbor relies on, seems to think it should be rotated:
The files listed in the task description have now been fixed. If you encounter more, try to purge them and shift-refresh/clear your browser cache.
Similar symptoms, but doesn't seem like a duplicate. The fix for T173804 worked for the files listed there, but not for this one.
Sorry, we keep forgetting to acknowledge those in Icinga because the emails and IRC notifications are enough for us. This alert is well explained, it's the result of comparing to older data, in this case comparing to a period where our metric collection had issues. I wish there was a way to auto-acknowledge alerts in Icinga, because we really don't need that part, the notifications are enough for our need.
Hah, it's been working the entire time. Of course it doesn't work if you hit Thumbor directly, only if you hit nginx:
Forks with the required changes:
I think we'll need to fork, some of them might make things insecure.
Why not both? And then once we've had them running on Wikimedia Cloud we can see which one is more stable, convenient, etc.
Fri, Sep 22
Thu, Sep 21
Very interesting talk about recent original research on perceived performance on pageload: https://air.mozilla.org/measuring-the-subjective-the-performance-dashboard-with-estelle-weyl/
Wed, Sep 20
Tue, Sep 19
We're going to need 3d2png deployed on the Thumbor servers. I don't know how that's done, though.
Something worth discussing here is whether the page queue *should* make visual progression worse. After all from the code's perspective, we already loading things low priority. But there's no option offered (out of the box) to load something low priority while being sure that it's not going to make the initial pageload worse in any way.
Unfortunately due to Mediawiki's filtering of EXIF before it reaches the image table, there's no way for us to compile a list of potentially affected files. Since this code has been running for 3 months now, purging all generated JPG thumbnails over that period is impractical. Which means that once the fix is deployed, affected files will have to be purged manually when encountered. Furthermore, once they fall out of Varnish cache eventually, they will fix themselves.
Should this be merged with T175935: Track visualComplete95 and visualComplete99 to make it easier to see when banners change SpeedIndex?
Mon, Sep 18
Default orientation EXIF field, seemingly conflicting width/height fields:
Coal is fine, it's the EL/kafka intake that seems to be stuck on the old schema.
Code that needs updating for the new metric:
SELECT COUNT(*) FROM `NavigationTiming_16305090` WHERE timestamp >= 20170912000000 AND timestamp < 20170913000000
Maybe the ingestion point looks at the old EL schema? I'm going to check EL SQL data to see if it's working there or not.
And the report rate confirms it, it tanked:
It seems like median/p75/p95 are not changed on the fly,
see that they keep the same value for hours (https://grafana.wikimedia.org/dashboard/db/navigation-timing-alerts?refresh=5m&orgId=1&panelId=23&fullscreen&from=1505436334194&to=1505598247554):
Thu, Sep 14
For later reference, with popups, 70% of pageloads result in a link interaction:
Experimenting locally, I can trigger it when opening VE on a cold cache, but not with a warm cache.
Wed, Sep 13
.git/info/exclude won't work with tracked files, but there is a workaround described on https://stackoverflow.com/questions/10879783/git-doesnt-ignore-2-specifically-named-files
Awesome, thank you!
I would like to keep this open, since it sounds like you're testing drastic changes to the UX, which should warrant another pass of performance review once they're done. Plus it sounds like the new version being tested might not replace the original, depending on the outcome of the test?
Tue, Sep 12
We've decided to follow the last recommendation and make it opt-in (beta feature for the foreseeable future), with the sidebar link.
We tried it for the Asia metrics and it sucked, huge variations between runs, completely unusable.
And I guess we can do the cleanup "just" for file types that can be multipage (TIFF, DJVU, PDF), as the header size isn't problematic for single page documents.
I think this meta task isn't that useful anymore, now that all the subtasks have been done except one.
We've abandoned X-Content-Dimensions, so I think we need to look at how we can clean it up. Is there a cheap way to find swift objects that have it set, or are we condemned to look at all originals?
Is this still a problem?
The subtasks are enough to keep track of that work.