There might be a delay between Nginx shipping a request to Thumbor and Thumbor picking it up, which could explain Nginx timeouts. Recording the request time from Nginx's POV should allow us to figure that out.
Description
Description
Details
Details
Related Changes in Gerrit:
Customize query in gerrit
Revisions and Commits
Revisions and Commits
| rTHMBREXT Thumbor Plugins | |||
| Restricted Differential Revision | rTHMBREXTb8d01d672299 Include request date header given by nginx in the response | ||
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • Gilles | T121388 Service-based thumbnailing re-architecture in production with Thumbor | |||
| Resolved | • Gilles | T150746 Nginx timeouts on Thumbor | |||
| Resolved | • Gilles | T161535 Track nginx request time in Thumbor debug headers |
Event Timeline
Comment Actions
Change 344968 had a related patch set uploaded (by Gilles):
[operations/puppet@production] Set nginx request time as a header passed to Thumbor
Comment Actions
Change 344968 merged by Filippo Giunchedi:
[operations/puppet@production] Set nginx request time as a header passed to Thumbor
Comment Actions
Change 345283 had a related patch set uploaded (by Gilles):
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 0.1.37
Comment Actions
Change 345283 merged by Filippo Giunchedi:
[operations/debs/python-thumbor-wikimedia@master] Upgrade to 0.1.37
Comment Actions
Doesn't seem to work right:
gilles@thumbor1001:~$ curl -I http://localhost:8809/wikipedia/en/thumb/9/9f/WUVI.jpg/720px-WUVI.jpg
HTTP/1.1 200 OK
Engine: wikimedia_thumbor.engine.imagemagick
Content-Length: 76102
Processing-Utime: 412
Content-Disposition: inline;filename*=UTF-8''WUVI.jpg
Wikimedia-Original-Container: wikipedia-en-local-public.9f
Request-Date: None
Thumbor-Parameters: {"width": "720", "image": "http://ms-fe.svc.eqiad.wmnet/v1/AUTH_mw/wikipedia-en-local-public.9f/9/9f/WUVI.jpg", "filters": "conditional_sharpen(0.0,0.8,1.0,0.0,0.85):format(jpg)"}
Server: Thumbor/6.0.1
Wikimedia-Thumbnail-Container: wikipedia-en-local-thumb.9f
Etag: "d4ead8672c3091844afb4b162c4cf72d6a0af3a0"
Processing-Time: 453
Convert-Time: 72
Date: Thu, 30 Mar 2017 16:21:35 GMT
Xkey: File:WUVI.jpg
Wikimedia-Original-Path: 9/9f/WUVI.jpg
Content-Type: image/jpeg
Exiftool-Time: 109
Exiftool-Time: 128
Exiftool-Time: 139
Wikimedia-Thumbnail-Path: thumbor/9/9f/WUVI.jpg/720px-WUVI.jpgComment Actions
Hah, it's been working the entire time. Of course it doesn't work if you hit Thumbor directly, only if you hit nginx:
gilles@thumbor1001:~$ curl -I http://localhost:8800/wikipedia/en/thumb/9/9f/WUVI.jpg/720px-WUVI.jpg
HTTP/1.1 200 OK
Server: nginx/1.11.13
Date: Mon, 25 Sep 2017 11:02:31 GMT
Content-Type: image/jpeg
Content-Length: 76102
Connection: close
Swift-Time: 83
Content-Disposition: inline;filename*=UTF-8''WUVI.jpg
Wikimedia-Original-Container: wikipedia-en-local-public.9f
Request-Date: Monday, 25-Sep-2017 11:02:31 GMT
Thumbor-Parameters: {"width": "720", "image": "https://ms-fe.svc.eqiad.wmnet/v1/AUTH_mw/wikipedia-en-local-public.9f/9/9f/WUVI.jpg", "filters": "conditional_sharpen(0.0,0.8,1.0,0.0,0.85):format(jpg)"}
Wikimedia-Thumbnail-Container: wikipedia-en-local-thumb.9f
Etag: "d4ead8672c3091844afb4b162c4cf72d6a0af3a0"
Xkey: File:WUVI.jpg
Wikimedia-Original-Path: 9/9f/WUVI.jpg
Wikimedia-Thumbnail-Path: 9/9f/WUVI.jpg/720px-WUVI.jpg
X-Upstream: 127.0.0.1:8837Comment Actions
Change 380480 had a related patch set uploaded (by Gilles; owner: Gilles):
[mediawiki/vagrant@master] Pass request time from nginx to thumbor
Comment Actions
Change 380480 merged by jenkins-bot:
[mediawiki/vagrant@master] Pass request time from nginx to thumbor
Comment Actions
Change 380483 had a related patch set uploaded (by Gilles; owner: Gilles):
[operations/puppet@production] Thumbor: expose Nginx request time
Comment Actions
Change 380483 merged by Filippo Giunchedi:
[operations/puppet@production] Thumbor: expose Nginx request time
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2017-09-26T08:09:53Z] <godog> roll-restart swift-proxy to apply https://gerrit.wikimedia.org/r/#/c/380483/ - T161535