Page MenuHomePhabricator

Eventgate error: '' should have required property 'database', '' should have required property 'performer'
Closed, ResolvedPublicPRODUCTION ERROR

Description

Error

single document: https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-default-1-7.0.0-1-2022.09.08?id=S17uHYMBzGlbejpUIo0n

normalized_message
'' should have required property 'database', '' should have required property 'performer'
exception.trace
Impact

~400-500 in past 12 hours at the time of writing

More than 32,000 in the last 24 hrs as of 9/9/23 UTC 15:33

~30,000 in the last 24 hrs as of 9/21/22 UTC 15:28 - https://logstash.wikimedia.org/goto/8f8b69836e510160c98319d22ed825b7

Notes

https://logstash.wikimedia.org/goto/bf44e9cea5784f98847e98cfbff283d4

Guessing error is related to this instrument that got deployed recently T306018

errored_stream_name: mediawiki.accountcreation_block

Event Timeline

+Event-Platform Value Stream (please add codebase project tags so such tasks show up on the appropriate workboards - thanks a lot!)

cjming updated the task description. (Show Details)
cjming added a subscriber: cjming.

Looks like these errors are still firing quite a bit -- added the Product Analytics + Growth Team tags because I think this is related to T306018 << recent patches should be on train (1.40.0-wmf.1)

Tgr added a subscriber: Tgr.

The patches were on wmf.29 (and the errors have been around for a while - I forgot to update our analytics error dashboard to include errors about this schema so we missed it) but otherwise yes, this is caused by the block metrics work. I guess I just assumed that fragment data would be added automagically, which would not make much sense in retrospect.

Change 833845 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[schemas/event/secondary@master] Un-require some analytics/mediawiki/accountcreation/block fields

https://gerrit.wikimedia.org/r/833845

Change 833846 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/WikimediaEvents@master] Block metrics: Bump schema to un-require some fields

https://gerrit.wikimedia.org/r/833846

Change 833845 merged by jenkins-bot:

[schemas/event/secondary@master] Un-require some analytics/mediawiki/accountcreation/block fields

https://gerrit.wikimedia.org/r/833845

Change 833846 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Block metrics: Bump schema to un-require some fields

https://gerrit.wikimedia.org/r/833846

Change 833809 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/WikimediaEvents@wmf/1.40.0-wmf.1] Block metrics: Bump schema to un-require some fields

https://gerrit.wikimedia.org/r/833809

Change 833810 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/WikimediaEvents@wmf/1.40.0-wmf.2] Block metrics: Bump schema to un-require some fields

https://gerrit.wikimedia.org/r/833810

Change 833809 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.40.0-wmf.1] Block metrics: Bump schema to un-require some fields

https://gerrit.wikimedia.org/r/833809

Change 833810 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@wmf/1.40.0-wmf.2] Block metrics: Bump schema to un-require some fields

https://gerrit.wikimedia.org/r/833810

Mentioned in SAL (#wikimedia-operations) [2022-09-21T20:36:53Z] <tgr@deploy1002> Synchronized php-1.40.0-wmf.1/extensions/WikimediaEvents/includes/BlockMetrics/BlockMetricsHooks.php: Backport: [[gerrit:833809|Block metrics: Bump schema to un-require some fields (T317343)]] (duration: 03m 55s)

Mentioned in SAL (#wikimedia-operations) [2022-09-21T20:44:58Z] <tgr@deploy1002> Synchronized php-1.40.0-wmf.2/extensions/WikimediaEvents/includes/BlockMetrics/BlockMetricsHooks.php: Backport: [[gerrit:833810|Block metrics: Bump schema to un-require some fields (T317343)]] (duration: 03m 42s)

Fixed & added the stream to the Growth eventlogging error dashboard.

Change 833857 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[schemas/event/secondary@master] analytics/mediawiki/accountcreation/block: Re-add required flags

https://gerrit.wikimedia.org/r/833857

Change 834338 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/WikimediaEvents@master] Block metrics: Add 'database' and 'performer' fields

https://gerrit.wikimedia.org/r/834338

Change 834339 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/EventBus@master] Add EventFactory::createMediaWikiCommonAttrs()

https://gerrit.wikimedia.org/r/834339

Change 833857 merged by jenkins-bot:

[schemas/event/secondary@master] analytics/mediawiki/accountcreation/block: Re-add required flags

https://gerrit.wikimedia.org/r/833857

Change 834339 merged by jenkins-bot:

[mediawiki/extensions/EventBus@master] Add EventFactory::createMediaWikiCommonAttrs()

https://gerrit.wikimedia.org/r/834339

Change 834338 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Block metrics: Add 'database' and 'performer' fields

https://gerrit.wikimedia.org/r/834338

Etonkovidova added a subscriber: Etonkovidova.

The error stopped at Sep 21, 2022 @ 20:43:33.068( logstash link)