Page MenuHomePhabricator

Applebot overloading all of tool labs
Closed, ResolvedPublic

Description

Hitting / repeatedly at > 1 request/second, effectively DOSing the front page.

17.142.152.10 - - [06/May/2016:16:46:20 +0000] "GET / HTTP/1.1" 200 93794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:21 +0000] "GET / HTTP/1.1" 200 93794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:21 +0000] "GET / HTTP/1.1" 200 93794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:23 +0000] "GET / HTTP/1.1" 200 93794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:27 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:32 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:33 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:34 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:38 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:39 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:39 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:39 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:40 +0000] "GET / HTTP/1.1" 200 16129 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:42 +0000] "GET / HTTP/1.1" 200 93792 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"

Event Timeline

Restricted Application added a project: Cloud-Services. · View Herald TranscriptMay 6 2016, 4:46 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald Transcript
valhallasw updated the task description. (Show Details)May 6 2016, 4:47 PM

Although they have stopped crawling / by themselves, they are still hitting other endpoints like crazy:

17.142.152.10 - - [06/May/2016:16:56:18 +0000] "GET /apple-app-site-association HTTP/1.1" 403 18 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:56:18 +0000] "GET /apple-app-site-association HTTP/1.1" 403 18 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:56:18 +0000] "GET /apple-app-site-association HTTP/1.1" 403 18 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)"

(the 403 is because of the ban, it 404ed before)

valhallasw closed this task as Resolved.May 27 2016, 1:04 PM
valhallasw claimed this task.
valhallasw added a subscriber: Andrew.

Banned (https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Banned); @Andrew emailed them but I don't think we have received a response, so applebot stays blocked for now.