This is a replacement for the existing kafka->librdkafka/kafkatee->log_file->django_banner_stats->pgehres-database pipeline.
Current kafka topic "webrequest_text is consumed and filtered into two buckets:
- landingpages - filtered for hostname "donate.wikimedia.org"
- beaconImpressions - filtered for string "beacon/impression" and sampled 1:10
There is/was an alternate kafka topic that FR no longer consumes that is based on eventlogging. This approach does not require filtering/sampling on the consumer side IIRC.
We will need to decide which to consume, and each approach is a compromise. The first (webrequest_text) may lose data during high traffic because of network limitations. The eventlogging based approach may undercount due to client side browser behavior. History and technical discussion of the different approaches can be found here T236835: FRUEC: Debug minor discrepancy in landing page data between old and new pipelines and more broadly here T183978: [Epic] Fundraising kafkatee changes.
Related: T253050: Bring Banner History data into Fundraising infrastructure