To test the fix locally:

• Set up two campaigns, one higher priority than the other.
• Include a close button in the banner for the campaign with higher priority.
• The higher priority campaign should always show, until the close button is clicked on the banner. On subsequent pageviews after the button has been clicked, the lower priority campaign should show.
• To test again, clear cookies for the local wiki.

In T240802#5797791, @mepps wrote:

@AndyRussG Okay, I can see what you mean. When I look at the loop what I see is that state.setCampaign is called early on and that should set the status to CAMPAIGN_CHOSEN for the campaign that gets selected. cancelBanner is called if hide.shouldHide() is true. Just recording my findings as I go.

In T240802#5790095, @mepps wrote:

@AndyRussG Is there any documentation of what these status codes mean?

Here's an overview of the status of this project:

Just updated the draft specification for this. We should also look at legacy default values for empty country and project fields.

In T236834#5737459, @Jgreen wrote:

We've set up a new kafkatee collector host at codfw, frban2001, which will replace the aged alnitak once we're confident everything is working properly on Debian Buster etc. In the meantime it's useful for comparison, to help rule out losses in the kafka->kafkatee->logfile segment in the pipeline. I ran all of yesterdays new-pipeline landingpages json logs through a simple perl json decoder and observed that alnitak and frban2001 collected exactly the same data for 12/11. The script I used is at mintaka:/tmp/json-log-filter, and frban2001's log store is mounted at mintaka:/mnt/banner_logs_new. The beacon-impressions logs are harder to compare since they're sampled at 1:10, and the two hosts are expected to collect different messages. So we set frban1001 to collecting beacon-impressions 1:1 for a day, and at the end we'll compare what was collected to what's in Hive.

As per T219604, it's just the RL modules aliases that have changed, but the modules are still around, so we should be able to fix pretty easily.

In T240505#5733732, @Jdforrester-WMF wrote:

jquery.ui.datepicker was removed in wmf.10, yes. wmf_deploy strikes yet again.

In T239778#5715514, @sbassett wrote:

Ugh, thanks for finding that, @Bawolff. I'm going to break that out into a separate high/ubn ticket.
Filed: T239919

@Bawolff thank you so much for finding this!!!! This same code is in live FR banners across enwiki. I've created a separate task: T239922

Hi! Here's my request for the new creds for stat100* and notebook100*, please. Username: andyrussg. Thanks so much for working on this!!!!! :)

I think it's blocking on the URL path /beacon/event. See https://easylist.to/easylist/easyprivacy.txt and T220627#5638168.

Just to note, we have the same problem for the new CentralNotice data pipeline, which uses EventLogging, as compared to the old pipeline, which uses a custom call to beacon/impression not blocked by AdBlock. In case it's useful: see T236834#5696044 (and the two comments after that).

In T236835#5682487, @Isaac wrote:

I'm not sure if this is at all pertinent, but we spent some time trying to debug a situation where we were missing about 10% of EventLogging data from people taking a survey served via the QuickSurveys tool. In essence, for 10% of readers, the tool was displaying the surveys correctly and we knew they had taken the survey but we never get the EventLogging that we should have. We ultimately decided that it was a mixture of adblock (which has settings that allow the javascript etc. to show the survey but blocks EventLogging) and, in our case, people could right-click off the page to take the survey and that wasn't triggered EventLogging as expected.
T220627#5641946

Windows for log samples:

Differences found in orphaned old pipeline events:

• 28% orphaned GET requests vs. 10% overall GET requests
• 63% orphaned Windows requests vs. 39% overall Windows requests

Just did one-to-one merges using web request logs in Hive, in both directions, using fairly large samples in both cases.

@elukey @Nuria Thanks so much!!!!!!!!!!!!!!!!!!!!!!!!

Here are some results:

In T238560#5682048, @Nuria wrote:

could you give some examples of issues you expect to see and troubleshoot (maybe some tickets from the past?)?

Hi all! Congrats to all for your work on this...

Windows for log samples:

Here's a summary of the situation regarding discrepancies between the new and old pipelines for Landing Pages (includes also measures obtained in T235284).

See also T237997: FRUEC: For legacy compatibility, empty language property should default to 'en' for LandingPage impressions.

See also T238023: FREUC: Check that empty or missing uselang or language URL parameter should default to 'en' language for Landing Pages.

FRUEC currently accepts LandingPage events with no language property in the JSON input. However, if the property exists and its value is an empty string, the event is marked invalid and not counted. However, in such a case, the legacy system (DjangoBannerStats) defaults the language to 'en'.