Page MenuHomePhabricator
Create Task
Maniphest T314173

editgrowthconfig schema: '' should NOT have additional properties,
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error
normalized_message
'' should NOT have additional properties, '' should NOT have additional properties
raw_event
{"meta":{"domain":"nn.wikipedia.org","stream":"mediawiki.editgrowthconfig","id":"<redacted>","dt":"2022-07-29T18:12:04.273Z","request_id":"55ad0103-7ce9-4843-bc0f-1bddfa13f2e0"},"http":{"request_headers":{"user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"}},"$schema":"/analytics/mediawiki/editgrowthconfig/1.0.0","database":"nnwiki","action":"view","is_privileged_user":false,"dt":"2022-07-29T18:12:04Z"}
Impact

~250 in last 12 hrs - https://logstash.wikimedia.org/goto/ab63971a6eb48b6ab6d8baa5f77f02f0

introduced 7/21/22 with a few errors, jumped on 7/27/22 - https://logstash.wikimedia.org/goto/7cd6ca6c82fc247c8dcf611ba1524f30

~1444 total since 7/21/22

Notes

meta.stream: eventgate-analytics-external.error.validation
type: eventgate_validation_error
errored_stream_name: mediawiki.editgrowthconfig

Details

SubjectRepoBranchLines +/-
SpecialEditGrowthConfigLogger: Update schema versionmediawiki/extensions/GrowthExperimentswmf/1.39.0-wmf.23+1 -1
SpecialEditGrowthConfigLogger: Update schema versionmediawiki/extensions/GrowthExperimentsmaster+1 -1
analytics/mediawiki/editgrowthconfig: Ref analytics/commonschemas/event/secondarymaster+338 -4
Customize query in gerrit

Related Objects
Search...

Event Timeline

cjming created this task.Jul 29 2022, 6:37 PM
Restricted Application added projects: Growth-Team, Data-Engineering, Community-Tech. · View Herald TranscriptJul 29 2022, 6:37 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
• EChetty moved this task from Incoming (new tickets) to Data Products & Metrics on the Data-Engineering board.Aug 1 2022, 1:08 PM
• EChetty edited projects, added Data-Engineering-Planning; removed Data-Engineering.Aug 1 2022, 1:11 PM
Restricted Application edited projects, added Data-Engineering; removed Data-Engineering-Planning. · View Herald TranscriptAug 1 2022, 1:11 PM
dmaza removed a project: Community-Tech.Aug 1 2022, 2:08 PM
Tgr edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.Aug 3 2022, 7:52 PM
Tgr added subscribers: Urbanecm_WMF, Tgr.

Whoops, looks like our eventlogging error dashboard was missing the mediawiki.editgrowthconfig stream. Now fixed.

@Urbanecm_WMF do you have any guess what might be going on? The error message isn't super helpful.

Tgr moved this task from Incoming to Ready for Development on the Growth-Team (Sprint 0 (Growth Team)) board.Aug 3 2022, 7:52 PM
Urbanecm_WMF claimed this task.Aug 3 2022, 8:27 PM
Urbanecm_WMF triaged this task as High priority.
Urbanecm_WMF edited projects, added GrowthExperiments-EditGrowthConfig, User-Urbanecm_WMF (Engineering); removed GrowthExperiments.
In T314173#8129392, @Tgr wrote:

Whoops, looks like our eventlogging error dashboard was missing the mediawiki.editgrowthconfig stream. Now fixed.

Didn't know that needs to be done when adding a stream. Will remember it for the next time.

@Urbanecm_WMF do you have any guess what might be going on? The error message isn't super helpful.

I probably did some mistake when working on T312148: Add instrumentation to Special:EditGrowthConfig. I'll have a look!

Urbanecm_WMF added a parent task: T312148: Add instrumentation to Special:EditGrowthConfig.Aug 3 2022, 8:27 PM
gerritbot added a comment.Aug 3 2022, 8:53 PM

Change 820227 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[schemas/event/secondary@master] analytics/mediawiki/editgrowthconfig: Ref analytics/common

https://gerrit.wikimedia.org/r/820227

gerritbot added a project: Patch-For-Review.Aug 3 2022, 8:53 PM

Change 820228 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] SpecialEditGrowthConfigLogger: Update schema version

https://gerrit.wikimedia.org/r/820228

Urbanecm_WMF moved this task from Ready for Development to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Aug 3 2022, 8:55 PM
In T314173#8129517, @Urbanecm_WMF wrote:
In T314173#8129392, @Tgr wrote:

@Urbanecm_WMF do you have any guess what might be going on? The error message isn't super helpful.

I probably did some mistake when working on T312148: Add instrumentation to Special:EditGrowthConfig. I'll have a look!

Indeed, I made an interesting mistake. I didn't add $ref: /fragment/analytics/common/2.0.0 to the schema (probably thinking that $ref: /fragment/mediawiki/common/2.0.0 will include analytics/common for me). That resulted in the 1.0.0 schema not having the top-level http field (compare with a different, valid, schema). EventStreams added the http field regardless (as it expects the schema to know about it), which resulted in validation errors.

Patches uploaded. Sorry for the inconvenience!

Tgr moved this task from Code Review to QA on the Growth-Team (Sprint 0 (Growth Team)) board.Aug 4 2022, 1:35 AM

Thanks for the quick fix!

gerritbot added a comment.Aug 4 2022, 1:35 AM

Change 820227 merged by jenkins-bot:

[schemas/event/secondary@master] analytics/mediawiki/editgrowthconfig: Ref analytics/common

https://gerrit.wikimedia.org/r/820227

gerritbot added a comment.Aug 4 2022, 7:31 AM

Change 820228 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] SpecialEditGrowthConfigLogger: Update schema version

https://gerrit.wikimedia.org/r/820228

Maintenance_bot removed a project: Patch-For-Review.Aug 4 2022, 8:31 AM
gerritbot added a comment.Aug 4 2022, 9:02 AM

Change 820276 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@wmf/1.39.0-wmf.23] SpecialEditGrowthConfigLogger: Update schema version

https://gerrit.wikimedia.org/r/820276

gerritbot added a project: Patch-For-Review.Aug 4 2022, 9:02 AM
Urbanecm_WMF mentioned this in T312148: Add instrumentation to Special:EditGrowthConfig.Aug 4 2022, 9:08 AM
gerritbot added a comment.Aug 4 2022, 9:26 AM

Change 820276 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.39.0-wmf.23] SpecialEditGrowthConfigLogger: Update schema version

https://gerrit.wikimedia.org/r/820276

Maintenance_bot removed a project: Patch-For-Review.Aug 4 2022, 9:31 AM
Stashbot added a comment.Aug 4 2022, 9:38 AM

Mentioned in SAL (#wikimedia-operations) [2022-08-04T09:38:50Z] <urbanecm@deploy1002> Synchronized php-1.39.0-wmf.23/extensions/GrowthExperiments/includes/EventLogging/SpecialEditGrowthConfigLogger.php: rEGREba67dd940217: SpecialEditGrowthConfigLogger: Update schema version (T314173, T312148) (duration: 03m 18s)

Etonkovidova subscribed.EditedAug 5 2022, 10:38 PM

The errors seem to stop - https://logstash.wikimedia.org/goto/6e75427fdc39f0ae9c241418bc7de928. Last timestamp - Aug 4, 2022 @ 19:39:16.373; will check in a couple of days before closing the ticket.

Checked on[[https://logstash.wikimedia.org/goto/6aece5a27ea8924290445fcbf56c8eea | Aug 08/2022]] - the error was not registered since Aug 4, 2022 @ 19:39:16.373.

Screen Shot 2022-08-08 at 6.44.48 PM.png (336×2 px, 59 KB)

Etonkovidova moved this task from QA to Test in Production | Watching on the Growth-Team (Sprint 0 (Growth Team)) board.Aug 5 2022, 10:43 PM
Ottomata removed projects: Data-Engineering, EventStreams.Aug 8 2022, 7:10 PM
Restricted Application added a project: Data-Engineering. · View Herald TranscriptAug 8 2022, 7:10 PM
Ottomata removed a project: Data-Engineering.Aug 8 2022, 7:10 PM
Restricted Application added a project: Data-Engineering. · View Herald TranscriptAug 8 2022, 7:10 PM
Ottomata removed a project: Data-Engineering.Aug 8 2022, 7:10 PM
Restricted Application added a project: Data-Engineering. · View Herald TranscriptAug 8 2022, 7:10 PM
Etonkovidova closed this task as Resolved.Aug 9 2022, 1:48 AM
Etonkovidova attached a referenced file: F35411954: Screen Shot 2022-08-08 at 6.44.48 PM.png. (Show Details)
Urbanecm_WMF removed projects: Data-Engineering, User-Urbanecm_WMF (Engineering), Event Metrics, MediaWiki-extensions-EventLogging.Aug 9 2022, 10:31 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits