Page MenuHomePhabricator

EventLogging MobileWebSectionUsage has backwards incompatible changes
Closed, InvalidPublic

Description

Recently some backwards incompatible changes were made to the MobileWebSectionUsage schema. In addition, an array field was added without the items type specified.

Removing required fields is not allowed, and adding new required fields is not allowed. The only schema changes that are allowed are to add new optional fields.

I just edited https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/Schema_Guidelines#Schema_set_up to indicate that array items are required. Otherwise we can't create Hive tables as we don't know the type of the array items, and get errors like:

RefineTargetException: Failed refinement of hdfs://analytics-hadoop/wmf/data/raw/eventlogging/eventlogging_MobileWebSectionUsage/hourly/2019/09/03/14 -> `event`.`MobileWebSectionUsage` (year=2019,month=9,day=3,hour=14). Original exception: java.lang.IllegalArgumentException: `initiallyOpenSections` array schema did not specify the items field)

Event Timeline

Ottomata created this task.Sep 3 2019, 8:28 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 3 2019, 8:28 PM

MobileWebSectionUsage is disabled and has been disabled for some time. How is this event firing?

Schema was deactivated in January 2016 in T124220

Ottomata closed this task as Invalid.Sep 3 2019, 9:07 PM

Oh sorry! I just saw the 'August' date on those schema changes and assumed it was this August!

If that is the case then I don't know! Apparently some event came through from somewhere! I will ignore.