Page MenuHomePhabricator

QuickSurveyInitiation events are being dropped due to validation errors
Closed, DuplicatePublicBUG REPORT

Description

Over a million events have been dropped in the last 4 days due to the error "'.event.performanceNow' should be integer "
The increase is likely due to a survey being enabled.

  • We should disable the survey temporarily
  • Fix the issue
  • Restore the survey

https://logstash.wikimedia.org/goto/6e1aff14c46a82c77cca7e3de5437d04

Screen Shot 2022-04-05 at 12.19.44 PM.png (308×1 px, 58 KB)

Event Timeline

Jdlrobson renamed this task from QuickSurveyInitiation events are being dropped. to QuickSurveyInitiation events are being dropped due to validation errors.Apr 5 2022, 7:19 PM
Jdlrobson triaged this task as Unbreak Now! priority.
Jdlrobson updated the task description. (Show Details)

Looking into this it seems we attempt to send two events - one in resources/ext.quicksurveys.lib/lib.js and resources/ext.quicksurveys.lib/vue/QuickSurveyLogger.js
Presumably the former is outdated and not needed.

please see patch https://phabricator.wikimedia.org/T305171 both events are needed. We mistakenly changed the datatype that does not match schema and the aforementioned patch addresses this issue.

Yep can confirm that https://gerrit.wikimedia.org/r/c/mediawiki/extensions/QuickSurveys/+/768194 should fix this. Could this be backported at the earliest opportunity? The number of validation errors is very high.

Greetings Jdlrobson it appears this patch was merged in group0 train yesterday if I am not mistaken.

It was but we're seeing a very high volume of errors on group1 and group2 wikis: https://logstash.wikimedia.org/app/dashboards#/view/AXN5OoJu3_NNwgAUlbUT
Rather than wait for the train to roll out to everyone (at earliest Thursday) I'd recommend addressing this earlier.