Page MenuHomePhabricator

Script/bot rapidly hitting Pageviews tool
Closed, InvalidPublic

Description

The Pageviews Analysis tool has features automated error reporting, and since early morning August 10 (GMT) I'm seeing errors like this every other minute, with occasional breaks. This is apparently a bot or script, making requests for the same article with the same date range, etc. My theory is they are loading the page more than 10 times per second, so the pageviews API errors out due to throttling, and some weirdness causes the JavaScript to fail when drawing the chart, and the try/catch there will create the Phabricator paste.

Their user agent, as collected by JavaScript (too generic to block):
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:48.0) Gecko/20100101 Firefox/48.0

Then the most recent error shows they changed to Windows:
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:48.0) Gecko/20100101 Firefox/48.0

Anyway the tool is not broken; When I try loading the requested URL in Firefox 48 on OS X 10.11 it works:
https://tools.wmflabs.org/pageviews/?project=en.wikipedia.org&platform=all-access&agent=user&start=2016-08-02&end=2016-08-09&pages=Shahram_Amiri

...hence my theory this is a bot or automated script running a high-rate.

A weird unrelated issue is the access.log on Tool Labs isn't being written to for any requests, so I don't have any output to share from that file.