Analyse possible bot traffic for ptwiki article Ambev
Open, HighPublic
Actions

Assigned To

None

Authored By

	TiagoLubiana
	May 10 2021, 11:24 PM

Description

The article https://pt.wikipedia.org/wiki/AMBEV is constantly among the top 3 most accessed pages in ptwiki.

There was a jump in views in october 2020, with a 30-fold access rate, which kept consistent for many months.

It is a page for a company, so there is always the possibility of marketing strategy via the web app, or similar.
The page for Cleópatra show a similar jump around the same time.

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T138207 [Open question] Improve bot identification at scale
		Open		None	T282502 Analyse possible bot traffic for ptwiki article Ambev

Event Timeline

TiagoLubiana created this task.May 10 2021, 11:24 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 10 2021, 11:24 PM

I quickly played with the more detailed data in Turnillo for those two pages. The request look to come from different IP addresses and user agents, and the agents look like a proper desktop/mobile agents.

For the records, https://w.wiki/3cXe [restricted link] is the query I used.

@Urbanecm thanks for taking a look at it. It is still weird though, I follow the Wikipedia app for about 6 different languages, and I've not seen any other items with similar patterns. They always are top-ranked in ptwiki, no matter what happens on the week.

The page for YouTube on ptwiki has developed a similar pattern over the last month too...

That's about the same story :/. It has plausible user agents (nothing like windows 10 on the mobile domain), plausible ISPs (they match distribution across all of ptwiki, plus match market share rights well). The IPs are also quite unique.

I agree this kind of behavior at multiple pages is weird, but unfortunately I was unable to find something bot-like :/. Tagging Analytics in case they want to take another look, maybe I missed something obvious.

odimitrijevic assigned this task to Milimetric.Jul 12 2021, 4:13 PM

odimitrijevic triaged this task as High priority.

odimitrijevic added projects: Product-Analytics, Analytics-Kanban.

odimitrijevic moved this task from Incoming to Ops Week on the Analytics board.

ldelench_wmf moved this task from Triage to Tracking on the Product-Analytics board.Jul 13 2021, 5:09 PM

cchen subscribed.Jul 13 2021, 5:10 PM

Mayakp.wiki subscribed.Jul 13 2021, 6:10 PM

I took a quick look and I agree the requests seem diverse enough to be organic, even if they're really not. The only thing of note I found is that for YouTube, the pattern doesn't show up on the .m domain: https://w.wiki/3dYA. I'm also going to sweep this under the T138207 rug, just to keep all the possible examples in one place.

Milimetric added a parent task: T138207: [Open question] Improve bot identification at scale.Jul 13 2021, 8:45 PM

Milimetric removed a project: Analytics-Kanban.Jul 14 2021, 4:09 PM

(TODO) We could use a standard way of handling these tasks at WMF. I feel like it would be ignored if it was on a wiki, but maybe everyone that matters is cc-ed here so...

data anomaly that tickles our bot/automata spidey-sense happens
who should get tagged first? product-analytics? analytics?
quick initial analysis (inspired by @Urbanecm's thoughts above, @JAllemandou do you have other tricks and things you look for?)
- Purpose: try to find some simple heuristic we can easily add to our current bot classifier
- # check webrequest sampled in Turnilo, split by different dimensions to see if the bump is exclusive to a specific dimension value (eg. specific country)
- # look at UA qualitatively (do they look real and normal?)
- # look at IP and UA diversity
- # look at ISP distribution comparing with the overall wiki project
until T138207 is resolved, we can keep adding examples as subtasks there