Page MenuHomePhabricator

[Epic] Fundraising kafkatee changes
Open, MediumPublic

Details

Related Gerrit Patches:
operations/mediawiki-config : masterCentralNotice EventLogging banner impression data test

Related Objects

StatusSubtypeAssignedTask
OpenEjegg
ResolvedJgreen
ResolvedJgreen
Declined cwdent
ResolvedNone
ResolvedNone
ResolvedAndyRussG
ResolvedAndyRussG
InvalidNone
OpenNone
ResolvedAndyRussG
ResolvedAndyRussG
OpenAndyRussG
ResolvedAndyRussG
ResolvedJgreen
OpenNone
Openjgleeson
OpenNone
OpenNone
OpenNone
OpenNone
DuplicateAndyRussG
OpenNone
ResolvedAndyRussG
OpenNone
InvalidAndyRussG
OpenNone
OpenNone
OpenAndyRussG
OpenNone
ResolvedAndyRussG
ResolvedJgreen
ResolvedNone
ResolvedAndyRussG
ResolvedAndyRussG
ResolvedAndyRussG
ResolvedAndyRussG
ResolvedAndyRussG
ResolvedAndyRussG
OpenAndyRussG
OpenNone
ResolvedAndyRussG
OpenNone
ResolvedAndyRussG
OpenNone
OpenNone
OpenNone
OpenNone
OpenAndyRussG
OpenAndyRussG
OpenNone
OpenNone

Event Timeline

DStrine created this task.Jan 2 2018, 7:25 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 2 2018, 7:25 PM
Ejegg claimed this task.Jan 19 2018, 9:14 PM
Ejegg triaged this task as Medium priority.
DStrine renamed this task from Make phab tasks for Kafkatee changes to [Epic] Kafkatee changes.Feb 13 2018, 9:07 PM
DStrine added a project: Epic.

Change 430613 had a related patch set uploaded (by AndyRussG; owner: AndyRussG):
[operations/mediawiki-config@master] CentralNotice EventLogging banner impression data test

https://gerrit.wikimedia.org/r/430613

Change 430613 merged by jenkins-bot:
[operations/mediawiki-config@master] CentralNotice EventLogging banner impression data test

https://gerrit.wikimedia.org/r/430613

Mentioned in SAL (#wikimedia-operations) [2018-05-03T19:01:58Z] <thcipriani@tin> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:430613|CentralNotice EventLogging banner impression data test]] T183978 (duration: 01m 04s)

Mentioned in SAL (#wikimedia-operations) [2018-05-04T00:55:09Z] <niharika29@tin> Synchronized wmf-config/CommonSettings.php: Temporary low-level activation of Eventlogging impression data for testing T183978 (duration: 01m 16s)

Jgreen renamed this task from [Epic] Kafkatee changes to [Epic] Fundraising kafkatee changes.Aug 21 2018, 3:39 PM

As of this writing, the open tasks attached to this epic do approximate what needs to be done. Probably as we move forward we'll think of more specific ones related to parallel deployment of the new system and switching off of the old one.

What the epic doesn't show is which tasks already have some progress, and what the priorities are. Here are some tasks that are already have some progress. All of these tasks are currently in review.

I'd suggest first we finish there review on these tasks, verify the data on the part of the pipeline that is already active (EventLogging data), then maybe plan more details of the deployment process.

Also quite a bit of work has been done by Analytics folks towards the new Druid dataset (for Turnilo and Superset Web interfaces). See the tasks listed here: T186048#5205429.

Here's an overview of the status of this project:

  • Data specification for the new pipeline is ready, except for possible adjustments for empty LandingPage values (see below).
  • Log files are created from the new Kafka streams.
  • Parallel testing database is on production.
  • FRUEC (new ingress script) is functional and can be deployed and run as a job on production.
  • No automated tests or CI yet.
  • No attempts yet to run real queries for Advancement on data from the new pipeline.
  • Major issue requiring attention: discrepancies between data in the old and new pipelines.

Details on discrepancies between new and old data pipelines:

  • No significant discrepancies detected due to FRUEC script.
    • One minor, easily fixable issues has been found: different behaviour for empty string language code. See T237997.
    • Other similar, minor issues should be checked for. See T242065.
  • Likely no major issues with Kafkatee filtering of events received from Analytics' Kafka streams. Task about further verification: T242022.
  • Major issues exist regarding the initial sections of the pipelines, that is, how data is initially collected from the client.
    • For the CN (banner) pipeline, both old and new events are generated client-side.
      • There are 12-15% less events in the new CN pipeline, apparently due to some adblock filters blocking all client-side MW EventLogging events.
      • Analysis of data: T236834.
      • What to do: T239570.
    • For the Landing Page pipeline, the old events are generated from requests for the base page HTML, and new events are generated client-side.
      • There are many more events in the old LandingPage pipeline, probably due to bots, spiders, hacking attempts, and proxies for web accelerators that don't run JavaScript. This is unavoidable but OK, since it's likely that we don't actually want to count all or nearly all of these extra events.
      • Strangely, there are also consistently extra events in the new LandingPage pipeline. These are events in the new pipeline that don't have a corresponding event in the old pipeline. Try to figure out what's going on: T238592.
      • Since the new pipeline events arrive on the same EventLogging URL, it is likely that 12-15% are being blocked by adblock filters, just like CN pipeline events.
      • Analysis of data: T236835.
      • What to do: T237553.