Page MenuHomePhabricator

HTTP/2 Issue with OkHttp/nginx
Closed, ResolvedPublic

Description

An implementation issue (arguably a bug) in how nginx 1.9.x+ negotiates connections, and OkHttp's failure to recover from it, currently prevent the Android app from logging in or editing over an HTTP/2 connection. We've worked around this for the time being by forcing OkHttp to use HTTP/1.1. Meanwhile, nginx is working to modify this behavior and OkHttp is working on a patch to gracefully recover from the connection failure, to be included in the next release (3.3). We should monitor both so that we can benefit from HTTP/2 when a fix is released.

Details here:
https://trac.nginx.org/nginx/ticket/959
https://lists.w3.org/Archives/Public/ietf-http-wg/2016AprJun/0174.html

See also discussion on the OkHttp repo:
https://github.com/square/okhttp/issues/2506
https://github.com/square/okhttp/issues/2543

And related bugs triggered by the HTTP/2 rollout:
T134758: [BUG] Unable to load Edit preview, and thus unable to save edit.
T134759: [BUG] Unable to log into the app.

Details

Related Gerrit Patches:

Event Timeline

Mholloway created this task.May 9 2016, 9:23 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptMay 9 2016, 9:23 PM
Mholloway updated the task description. (Show Details)May 9 2016, 9:24 PM
Mholloway moved this task from Needs Triage to Tracking on the Wikipedia-Android-App-Backlog board.
Mholloway updated the task description. (Show Details)May 9 2016, 11:15 PM
Mholloway triaged this task as Medium priority.May 9 2016, 11:18 PM

The relevant nginx ticket, with a proposed fix, is here:

https://trac.nginx.org/nginx/ticket/959

Reportedly, the Square folks have tested it and it works perfectly with OkHttp. (Not sure whether this will also solve the Firefox issue or that's entirely separate.)

Mholloway renamed this task from [Tracking] HTTP/2 Bug in OkHttp to [Tracking] HTTP/2 Issue with OkHttp/nginx.May 21 2016, 4:20 PM
Mholloway updated the task description. (Show Details)
Mholloway updated the task description. (Show Details)May 21 2016, 4:37 PM
Mholloway updated the task description. (Show Details)May 21 2016, 4:43 PM
Mholloway updated the task description. (Show Details)May 22 2016, 2:29 PM
Mholloway updated the task description. (Show Details)May 22 2016, 2:34 PM
Mholloway updated the task description. (Show Details)
Danny_B renamed this task from [Tracking] HTTP/2 Issue with OkHttp/nginx to HTTP/2 Issue with OkHttp/nginx (tracking).May 23 2016, 12:15 PM
Danny_B added a project: Tracking-Neverending.
Mholloway added a subscriber: BBlack.EditedJun 15 2016, 4:14 PM

OkHttp can recover from REFUSED_STREAM errors in HTTP/2 and downgrade appropriately as of v3.3.0, and we've upgraded so that we'll no longer crash when this happens while attempting an HTTP/2 connection. But OkHttp will just fall back and won't actually be able to connect over HTTP/2 until our nginx is upgraded to 1.11.

@BBlack, just for tracking purposes, when would you expect that to happen? Do we upgrade pretty regularly?

[1] https://nginx.org/en/CHANGES :

Changes with nginx 1.11.0                                        24 May 2016
[...]
*) Change: HTTP/2 clients can now start sending request body
   immediately; the "http2_body_preread_size" directive controls size of
   the buffer used before nginx will start reading client request body.

We run custom nginx packages and update pretty frequently. We deployed the change above (HTTP/2 body pre-read) back on June 1st.

Change 294306 had a related patch set uploaded (by Mholloway):
Remove custom protocol list from OkHttpConnectionFactory

https://gerrit.wikimedia.org/r/294306

Change 294306 merged by jenkins-bot:
Remove custom protocol list from OkHttpConnectionFactory

https://gerrit.wikimedia.org/r/294306

Mholloway closed this task as Resolved.Jun 28 2016, 8:21 PM
Danny_B renamed this task from HTTP/2 Issue with OkHttp/nginx (tracking) to HTTP/2 Issue with OkHttp/nginx.Jul 11 2016, 6:33 PM
Danny_B removed a project: Tracking-Neverending.