As expected this https://superset.wikimedia.org/r/14 query matches turnillo

Will look into this in the next couple days.

We have looked into whether the source of discrepancy could be unvalid events and there does not seem to be any schema validation errors for this schema for the months of 9, 10.

Other possible cause could be that not all impressions are being sent by your EL code in the client (note that EL counts are lower than the ones that come from parsing the whole firehouse)

Questions:

• Are these banners shown to ALL browsers or only to the ones to whom we serve javascript?

Note EL code uses sendbeacon and an image pixel, for browsers that do not support send beacon the EL request will not be sent if the customer moves out of the page before synchronous ping happen.

Ping @AndyRussG regarding the question: "Are these banners shown to ALL browsers or only to the ones to whom we serve javascript?"

More investigations on this.
As @Nuria explained above, we think that the highest probability of problem is the client not managing to send the eventlogging event, for instance because the browser doesn't have javascript.
I have double checked data on the cluster and it goes in that sense.
I queried turnilo, eventlogging and webrequest for yesterday (Oct 17th 2018) on campaign ITwikicon (small volume).
What I find is that turnilo == webrequest(/beacon/impression) (expected), and that eventlogging == webrequest(/beacon/event) (expected as well). And as suspected, turnillo == /beacon/impression is a little higher than eventlogging == /beacon/event.
Repro for turnilo: here
Repro for eventlogging and webrequest, on stat100[4|5]:

spark2-shell --master yarn

// EventLogging data
spark.sql("select hour, COUNT(1) as c from event.centralnoticeimpression where year=2018 and month = 10 and day = 17 and event.campaign = 'ITwikicon' group by hour order by hour").collect.foreach(println)

// Webrequest Beacon Event
spark.sql("SELECT hour, count(1) FROM wmf.webrequest where webrequest_source = 'text' and year=  2018 and month = 10 and day = 17 and uri_path = '/beacon/event' and uri_query like '%CentralNoticeImpression%' and uri_query like '%ITwikicon%' group by hour order by hour").collect.foreach(println)

// Webrequest Beacon Impression
spark.sql("SELECT hour, count(1) FROM wmf.webrequest where webrequest_source = 'text' and year=  2018 and month = 10 and day = 17 and uri_path = '/beacon/impression' and uri_query like '%ITwikicon%' group by hour order by hour").collect.foreach(println)

Results:

Some more context, using our data we can see that you are not sending as many beacon events (/beacon/event URL) with eventlogging as impressions are recorded via data in webrequest table (/beacon/impression URL), then:

FR tech needs to verify what browsers are receiving banners: are ALL browsers receiving banners ALSO capable of sending a EL events? If the answer to this question is yes you might be looking at bug in your EL code or (if difference is small), to browsers that do not support sendBeacon and thus, have navigated away from the page in which the banner is shown before the event is triggered. Seems that some quality checking of your EL data is in order to make sure you are not missing browsers, etc.

Assigning ticket to FR tech, moving to radar.

@Nuria, @JAllemandou, @mforns thanks so much taking the time to check this out, much appreciated!!! We can dig into it further (as priorities allow). :)

@AndyRussG

Can you answer these questions:

• are all browsers receiving banners?
• are only js-enabled browsers receiving banners?

In T204396#4680574, @Nuria wrote:

Can you answer these questions

Hey... of course, apologies for the delays!!!

• are all browsers receiving banners?
• are only js-enabled browsers receiving banners?

Only JS-enabled browsers display CentralNotice banners. The system for selecting a banner to show to a given user relies on heavily on JavaScript. (See browser support matrix.)

@JAllemandou I think you're approach of comparing Hive tables directly is right on! I'll continue this approach, and look for possible causes in the client-side JS (again, as priorities allow). :)

@AndyRussG I would look at EL data and see if any browser is notably missing from the events you have sent and received (hive events table). If the browsers missing are notably the ones w/o support of SendBeacon then your issue is users navigating outside the page before events get sent, that means -by proxy- that they did not engaged with banner. If, however, there is no clear browser pattern to your missing data you probably have an issue with your instrumentation that you need to fix.

FYI that we are puting this on radar, this item is on FR court

Quick update here. We are still using the older pipeline for this season so this is not affecting us at the moment. We'll try to look at this again next year.

@DStrine can you clarify what is the old pipeline?

Kafkatee

@AndyRussG assures us this would not have impacted our campaign stats.