Page MenuHomePhabricator

Undetected automated agents on nl.wiktionary
Open, Needs TriagePublicBUG REPORT

Description

Data Platform Engineering Bug Report or Data Problem Form.

Please fill out the following
Please ensure you set priority

What kind of problem are you reporting?

  • Access related problem
  • Service related problem
  • Data related problem
For an access related problem
  • What is the system you are using?
  • What is the data or dashboard you are unable to access? Please include links and screenshots where applicable.
  • What is your ldap user name? If you don’t know, please see the list https://ldap.toolforge.org/group/wmf. If you name is not on the list, use the following instructions to request an ldap account. In the form specify “wmf” as the ldap group name for staff as the ldap group name.or go through the nda process for external users. For Superset and Turnillo access, you do not need shell access, for Jupyter, Scripting and Hadoop you do.
For a service related problem:
  • What is the nature of the issue?
  • What are the steps to reproduce the issue? Please include links and screenshots where applicable.
  • What happens?
  • What should happen instead?
For a data related problem:
  • Is this a data quality issue?

Yes.

  • What datasets and/or dashboards are affected?

https://stats.wikimedia.org/#/nl.wiktionary.org/reading/total-page-views/normal|bar|1-month|agent~user|daily

  • What are the observed vs expected results? Please include information such as location of data, any initial assessments, sql statements, screenshots.

Starting Tuesday Jan 9th the nl.wiktionary user pageview count per day suddenly is nearly double the usual level. At the same time the ratio of those pageviews to unique devices grew from a little more than 2 to around 3. This sudden growth is only visible in the pageviews from desktops, pageviews from mobile are in a normal range. We have had peaks like this in the past for a few days, but by now the increase has been lasting for nearly two weeks.

Of course, we would be very happy if a small group of deeply interested desktop users suddenly has discovered us. However, before we start ordering the celebratory stroopwafels, I'd like to know whether we might be witnessing the birth of some kind of automated use you are not presently detecting?

I would be grateful if you could shine some light on this matter.

Thanks in advance!

For the DE Team to fill out
Which systems does this effect?
  • Hive
  • Druid
  • Superset
  • Turnilo
  • WikiDumps
  • Wikistats
  • Airflow
  • HDFS
  • Goblin
  • Scqoop
  • Dashiki
  • DataHub
  • Spark
  • Jupyter
  • Modern Event Platform
  • Event Logging
  • Other
Impact Assessment:

Does this problem qualify as an incident?

  • Yes
  • No

Does this violate an SLO?

  • Yes
  • No
Value CalculatorRank
Will this improve the efficiency of a teams workflow?1-3
Does this have an effect of our Core Metrics?1-3
Does this align with our strategic goals?1-3
Is this a blocker for another team?1-3

Event Timeline

I noticed a similar pattern starting Jan 9th on at least 6 other wiktionaries: fr el de sv ca it.

Wiktionaries with a sharp rise in pageviews at a later date: mg ku sh lt hu.

At some wiktionaries it's difficult to establish a significant change: en zh ru es pl.

@MarcoSwart thank you for filing this bug report. We will have a look and see if we can debug.

Reedy renamed this task from NEW BUG REPORT Undetected automated agents on nl.wiktionary to Undetected automated agents on nl.wiktionary.Jan 24 2024, 9:07 PM

After a quick look to nl.wiktionary pageviews, I agree that the anomalies are caused almost certainly by unidentified bots (automated traffic).
The requests come all from Chrome desktop, and are issued from a short list of Countries (Netherlands, Belgium and Singapore).

Moreover, the time-series pattern of correctly identified automated traffic mirrors the user traffic spikes brought up in this task.
This suggests that some of the automated requests were in fact identified as bots, but the majority were not.
This might happen for many reasons, but usually it's because the automated traffic comes from different IPs and UserAgents (i.e. botnet).
The more sophisticated the botnet, the more difficult it is for our automated traffic detection system to catch such requests.

I could not identify any specific pages that this bot action might have targeted to alter the top article rank.
Instead, I see that many pages have seen big increases in their pageview counts, many of them were already in the ranks of previous weeks.


I believe the solution to this is to improve our automated traffic detection.
This might seem obvious... What I mean is that there is no specific bug that we can fix now to solve this problem. But rather this is a general issue, that can only be mitigated with the overall improvement of the whole system.

I noticed that the increase in Pageviews is not accompanied by a similar increase in Unique Devices. For online dictionaries, Pageviews per visit (~unique device) are typically between 2 and 3. I don't know if we could somehow filter out very large (e.g. >100) numbers of pageviews from a single device.

Wiktionaries typically see a high number of pageviews from spiders and other bots compared with humans. The notion of improving the detection somewhere in the future is for this reason a little disconcerting. It means that the User statistic is becoming useless for comparisons over time. Or should we simply make a leap and just accept the total number of pageviews, automated or not, as the most relevant impact measurement for wiktionaries?