There is a spike of errors on eventlogging events that just triggered our alarms:
From brief inspection of log I see a lot of errors in TestSearchSatisfaction2, they will be present on events database on eventlogging on the eventerror schema table.
Ana example:
{"event": {"code": "validation", "message": "'isForced' is a required property", "rawEvent": "{\"dt\": \"2019-08-01T19:34:17Z\", \"event\": {\"action\": \"searchResultPage\", \
"articleId\": 2071, \"autocompleteType\": \"comp_suggest\", \"hitsReturned\": 10, \"inputLocation\": \"header\", \"msToDisplayResults\": 133, \"mwSessionId\": \"d5df422719645f8
fcee5\", \"pageViewId\": \"52c859335371401bf7fdjyt2vcjd\", \"query\": \"an\", \"scroll\": false, \"searchSessionId\": \"810b3490786ceebcdd79jyt29c47\", \"searchToken\": \"aoo4p
weec4h6vt11ofsgj35sq\", \"source\": \"autocomplete\", \"uniqueId\": \"30272db87b9aa4b3970ajyt2vd6h\"}, \"recvFrom\": \"cp1089.eqiad.wmnet\", \"revis
ion\": 19240637, \"schema\": \"TestSearchSatisfaction2\", \"seqId\": 61322129,
\"uuid\": \"94581eb6f5755cc8b910d57e17dec492\", \"webHost\": \"es.wikipedia.org\", \"wiki\": \"eswiki\"}", "revision": 19240637, "schema": "TestSearchSatisfaction2"}, "recvFro
m": "eventlog1002.eqiad.wmnet", "revision": 14035058, "schema": "EventError", "timestamp": 1564688057, "uuid": "59f3a124b49311e99e021418775b0d42", "wiki": ""}
{"event": {"code": "validation", "message": "'isForced' is a required property", "rawEvent": "{\"dt\": \"2019-08-01T19:34:16Z\", \"event\": {\"action\": \"click\", \"articleId\
": 54291073, \"mwSessionId\": \"849b7a0361159fe36652\", \"pageViewId\": \"8a8d4f95552440257936jyt2urk1\", \"position\": 1, \"scroll\": false, \"searchSessionId\": \"a6a6c7dc5ff
9d38c9f0ejyt18l0e\", \"searchToken\": \"arl87dzemdxoudbbyohwznia0\", \"source\": \"autocomplete\", \"uniqueId\": \"de6379cc02e7b4339e37jyt2uvcm\"}, \"ip\": \"47.197.5.246\", \"
recvFrom\": \"cp1083.eqiad.wmnet\", \"revision\": 19240637, \"schema\": \"TestSearchSatisfaction2\",
