Page MenuHomePhabricator

[EventGate] Add configurable UA denylist
Open, HighPublic5 Estimated Story Points

Description

Background

While investigating T427092: Run and synthetic A/A test that captures UA to investigate hoisting errors, I discovered that Bingbot accounts for a significant proportion of subject ID hoisting errors:

+-------+-----+------+-----------+-------------+
| month | day |  n   | n_bingbot | pct_bingbot |
+-------+-----+------+-----------+-------------+
|     5 |  27 |   28 |        12 |       42.86 |
|     5 |  28 | 1089 |       513 |       47.11 |
|     5 |  29 | 2906 |      2155 |       74.16 |
|     5 |  30 | 1819 |       890 |       48.93 |
|     5 |  31 | 1664 |       721 |       43.33 |
|     6 |   1 | 3489 |      2679 |       76.78 |
|     6 |   2 | 3012 |      2145 |       71.22 |
|     6 |   3 | 2086 |      1506 |        72.2 |
+-------+-----+------+-----------+-------------+

I presented the results to Experiment Platform and we agreed that we would like to be able to configure EventGate to quietly reject events from certain UAs.

AC

  • Add a UA denylist which causes EventGate to quietly reject events from certain UAs

Questions

  1. Should the UA denylist be configurable per-stream or global? If it was configurable per-stream, then we could use the $wgEventStreamsDefaultSettings to make a global one with per-stream overrides (maybe?)

Details

Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Add TestKitchen transformrepos/data-engineering/eventgate-wikimedia!35phuedxwork/phuedx/T430322master
Customize query in GitLab

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

If the goal is to reject events from self-declared bots, it's worth also keeping an eye on T430020, which is attempting to provide one canonical table for those.

If we can propagate that information to EventGate somehow, you won't need to maintain your own allowlist.

We may not want to blanket deny bots, but this one in particular is really impacting our SLO, which does suggest a targeted approach could work well.

For bots in the future, we'll want to leverage scoring rather than dropping information.

JVanderhoop-WMF added a project: Test Kitchen.
JVanderhoop-WMF moved this task from Incoming to Radar on the Test Kitchen board.
JVanderhoop-WMF moved this task from Radar to READY TO GROOM on the Test Kitchen board.
JVanderhoop-WMF added a subscriber: Ahoelzl.

@Ahoelzl @tchin -- this is impacting our SLO so we will tackle this and submit a patch for your review.

KReid-WMF set the point value for this task to 5.
KReid-WMF moved this task from READY TO GROOM to Backlog on the Test Kitchen board.