Swift invalid range requests causing 501s
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	fgiunchedi
	Jan 2 2018, 10:05 AM

Description

CC Traffic for visibility

It seems we're replying with 501 to requests to swift/varnish upload, where a 416 would be warranted instead, e.g. (I'm posting the logstash id for later reference)

GET https://upload.wikimedia.org/wikipedia/commons/6/6d/Elena_dementieva.ogg
bytes=18446744073709550047-
_id=AWC2Gg2dKr2U4sZ1omvc

Another source of 501s I've see is specifically from httrack, that seems to be requesting files with range=<last_byte>- (the file below has content-length 6379)

GET https://upload.wikimedia.org/wikipedia/commons/thumb/8/87/Kraków,_Widok_z_wieży_Kościoła_Mariackiego_-_fotopolska.eu_(220013).jpg/120px-Kraków,_Widok_z_wieży_Kościoła_Mariackiego_-_fotopolska.eu_(220013).jpg
bytes=6379-

See also swift tests for range requests at https://github.com/openstack/swift/blob/master/test/functional/tests.py#L1087
And the rfc for range requests for reference https://tools.ietf.org/html/rfc7233

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	swift: don't error on unknown HTTP status reason	operations/puppet	production	+7 -3
	swift: deprecate webob usage in rewrite.py	operations/puppet	production	+36 -51

Customize query in gerrit

Related Objects

Mentioned Here: T147162: upload.wikimedia.org returns HTTP 501 instead of 416 for non-satisfiable byte ranges

Event Timeline

fgiunchedi created this task.Jan 2 2018, 10:05 AM

Restricted Application added a project: SRE. · View Herald TranscriptJan 2 2018, 10:05 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

fgiunchedi updated the task description. (Show Details)Jan 2 2018, 10:12 AM

• ema moved this task from Backlog to Radar/Not for Service on the Traffic board.Jan 3 2018, 8:50 AM

Ottomata triaged this task as Medium priority.Jan 16 2018, 8:07 PM

fgiunchedi added a project: User-fgiunchedi.Jan 17 2018, 10:56 AM

As per related T147162: upload.wikimedia.org returns HTTP 501 instead of 416 for non-satisfiable byte ranges this is likely a bug in our usage of webob in rewrite.py. We should be using swob which is what swift itself uses (i.e. https://github.com/openstack/swift/blob/master/swift/common/swob.py)

fgiunchedi claimed this task.Apr 3 2018, 12:42 PM

fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.

Mentioned in SAL (#wikimedia-operations) [2018-04-03T15:09:27Z] <godog> depool ms-fe2005 to test rewrite.py - T183902

Mentioned in SAL (#wikimedia-operations) [2018-04-04T07:59:13Z] <godog> depool ms-fe2005 to test rewrite.py - T183902

Change 423852 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] swift: deprecate webob usage in rewrite.py

https://gerrit.wikimedia.org/r/423852

gerritbot added a project: Patch-For-Review.Apr 4 2018, 8:28 AM

Change 423852 merged by Filippo Giunchedi:
[operations/puppet@production] swift: deprecate webob usage in rewrite.py

https://gerrit.wikimedia.org/r/423852

Mentioned in SAL (#wikimedia-operations) [2018-04-04T13:56:37Z] <godog> rollout https://gerrit.wikimedia.org/r/c/423852 across ms-fe machines - T183902

Change 423937 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] swift: don't error on unknown HTTP status reason

https://gerrit.wikimedia.org/r/423937

Change 423937 merged by Filippo Giunchedi:
[operations/puppet@production] swift: don't error on unknown HTTP status reason