Page MenuHomePhabricator
Create Task
Maniphest T153699

Skewed pageviews for Azerbaijani and Bulgarian Wikipedias, September, October and November 2016
Closed, DuplicatePublic
Actions

Description

I see odd data for Azerbaijani and Bulgarian Wikipedias in the pageviews tool.

The most striking examples are from Azerbaijani:
https://tools.wmflabs.org/siteviews/?platform=all-access&source=pageviews&agent=user&start=2016-08-01&end=2016-12-18&sites=az.wikipedia.org

You can see a jump up starting towards the end of October.

The most popular page, by far, is called "xss":

https://tools.wmflabs.org/topviews/?project=az.wikipedia.org&platform=all-access&date=2016-11&excludes=

You can see something similar also in Bulgarian, although on a much more modest scale:

https://tools.wmflabs.org/siteviews/?platform=all-access&source=pageviews&agent=user&start=2016-08-01&end=2016-12-18&sites=bg.wikipedia.org

And there, too, "xss" is the most popular page by far, though not as extremely as in Azerbaijani:

https://tools.wmflabs.org/topviews/?project=bg.wikipedia.org&platform=all-access&date=2016-09&excludes=
https://tools.wmflabs.org/topviews/?project=bg.wikipedia.org&platform=all-access&date=2016-10&excludes=

I don't see anything similar in other languages.

In both languages, "xss" is a page that doesn't exist.

I noticed these two languages, because they appeared to have strangely skewed data about interlanguage links clicks, which I'm tracking regularly.

It would be nice to understand why does this happen.

This reminds me slightly of T141506, although the reason is possibly different.

Related Objects

Event Timeline

Amire80 created this task.Dec 19 2016, 5:59 PM
Restricted Application added a project: Analytics. · View Herald TranscriptDec 19 2016, 5:59 PM
Restricted Application added subscribers: kerberizer, Aklapper. · View Herald Transcript
Amire80 updated the task description. (Show Details)Dec 19 2016, 6:07 PM
• Nuria subscribed.EditedDec 20 2016, 8:46 PM

Amire80: from report it is likely a bot looking for an exploit

Mediawiki returns 200 for requests that should be 404s (pages that do not exist) and from our end is not possible to distinguish those from pageviews.

See: https://phabricator.wikimedia.org/T144100

• Nuria edited projects, added Analytics-Kanban; removed Analytics.Jan 23 2017, 4:47 PM
JAllemandou claimed this task.Jan 30 2017, 5:03 PM
JAllemandou moved this task from Next Up to In Progress on the Analytics-Kanban board.
JAllemandou added a comment.Jan 31 2017, 2:51 PM

More investigation:
Looking at December 2016 22nd:

These indeed fall into the big bucket of bots we don't catch :(

JAllemandou moved this task from In Progress to In Code Review on the Analytics-Kanban board.Jan 31 2017, 2:52 PM
• Nuria edited projects, added Analytics; removed Analytics-Kanban.Jan 31 2017, 9:11 PM

Untagging kanban. Adding as subtask of bot task

• Nuria added a parent task: T138207: [Open question] Improve bot identification at scale.Jan 31 2017, 9:12 PM
Milimetric moved this task from Incoming to Wikistats on the Analytics board.Feb 2 2017, 5:32 PM
• Tbayer mentioned this in T157528: Add "Damn Small XSS Scanner" (DSXS) to list of known bots.Feb 8 2017, 5:24 AM
• Tbayer subscribed.

Checking back for azwiki - see screenshot below - it appears to have ended there (exactly on christmas day BTW). But the culprit that @JAllemandou identified above is still generating an enormous amount of pageviews that are mistakenly classified as agent_type = 'user'. I have opened a new task about it, with more detail: T157528

az.wikipedia.org siteviews-20160701-20170206.png (598×1 px, 33 KB)

• Nuria added a comment.Feb 8 2017, 5:45 PM

@JAllemandou : sorry i should have looked into this closer, if this is a self-identified bot it should not be a subtask of this ticket, the cases we are agreggating here are the ones pertaining to bots that are not self-identified as such.

• Nuria removed a parent task: T138207: [Open question] Improve bot identification at scale.Feb 8 2017, 5:45 PM
• Nuria edited projects, added Analytics-Kanban; removed Analytics.Feb 8 2017, 5:47 PM
JAllemandou closed this task as a duplicate of T157528: Add "Damn Small XSS Scanner" (DSXS) to list of known bots.Feb 8 2017, 8:50 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits