Page MenuHomePhabricator
Create Task
Maniphest T134591

Applebot overloading all of tool labs
Closed, ResolvedPublic
Actions

Description

Hitting / repeatedly at > 1 request/second, effectively DOSing the front page.

17.142.152.10 - - [06/May/2016:16:46:20 +0000] "GET / HTTP/1.1" 200 93794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:21 +0000] "GET / HTTP/1.1" 200 93794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:21 +0000] "GET / HTTP/1.1" 200 93794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:23 +0000] "GET / HTTP/1.1" 200 93794 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:27 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:32 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:33 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:34 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:38 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:39 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:39 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:39 +0000] "GET / HTTP/1.1" 200 93790 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:40 +0000] "GET / HTTP/1.1" 200 16129 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:46:42 +0000] "GET / HTTP/1.1" 200 93792 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B410 Safari/600.1.4 (Applebot/0.1; +http://www.apple.com/go/applebot)"

Event Timeline

valhallasw created this task.May 6 2016, 4:46 PM
Restricted Application added a project: Cloud-Services. · View Herald TranscriptMay 6 2016, 4:46 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald Transcript
valhallasw updated the task description. (Show Details)May 6 2016, 4:47 PM
valhallasw added a comment.May 6 2016, 4:57 PM

Although they have stopped crawling / by themselves, they are still hitting other endpoints like crazy:

17.142.152.10 - - [06/May/2016:16:56:18 +0000] "GET /apple-app-site-association HTTP/1.1" 403 18 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:56:18 +0000] "GET /apple-app-site-association HTTP/1.1" 403 18 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)"
17.142.152.10 - - [06/May/2016:16:56:18 +0000] "GET /apple-app-site-association HTTP/1.1" 403 18 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Applebot/0.1; +http://www.apple.com/go/applebot)"

(the 403 is because of the ban, it 404ed before)

Matthewrbowker subscribed.May 6 2016, 5:23 PM
valhallasw closed this task as Resolved.May 27 2016, 1:04 PM
valhallasw claimed this task.
valhallasw added a subscriber: Andrew.

Banned (https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Banned); @Andrew emailed them but I don't think we have received a response, so applebot stays blocked for now.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL