Change 907508 had a related patch set uploaded (by Jameel Kaisar; author: Jameel Kaisar):

[schemas/event/primary@master] Created experimental/geoip/network_latency 1.0.0 schema

https://gerrit.wikimedia.org/r/907508

Change 907508 merged by Ottomata:

[schemas/event/primary@master] Created development/network/probe 1.0.0 schema

https://gerrit.wikimedia.org/r/907508

Change 913379 had a related patch set uploaded (by Jameel Kaisar; author: Jameel Kaisar):

[schemas/event/primary@master] Update development/network/probe 1.0.0 schem

https://gerrit.wikimedia.org/r/913379

Change 913394 had a related patch set uploaded (by Jameel Kaisar; author: Jameel Kaisar):

[mediawiki/extensions/WikimediaEvents@master] Add Probenet library and fetch code

https://gerrit.wikimedia.org/r/913394

Change 913379 merged by jenkins-bot:

[schemas/event/primary@master] Update development/network/probe 1.0.0 schem

https://gerrit.wikimedia.org/r/913379

Fixed eslint warnings in WikimediaEvents repo. This patch is not directly related to this task but the warnings were generated after the upgradation of eslint config in WikimediaEvents repo which was necessary for this task.
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/920376

Change 913394 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Add Probenet library and fetch code

https://gerrit.wikimedia.org/r/913394

Hey all, I think there are a bunch of validation errors on this stream:

https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-default-1-7.0.0-1-2023.06.12?id=l5Z9r4gB_1-qnHEYPZAS

https://prometheus-eqiad.wikimedia.org/k8s/classic/graph?g0.range_input=2h&g0.expr=sum%20by%20(stream)%20(rate(eventgate_validation_errors_total%7B%7D%5B15m%5D))&g0.tab=0

status_code should be a number?

There are enough that some alarms are triggering.

Change 929387 had a related patch set uploaded (by Jameel Kaisar; author: Jameel Kaisar):

[mediawiki/extensions/WikimediaEvents@master] Fix: Validation errors for status_code in Probenet reports

https://gerrit.wikimedia.org/r/929387

Change 929387 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Fix: Validation errors for status_code in Probenet reports

https://gerrit.wikimedia.org/r/929387

Change 930286 had a related patch set uploaded (by Jameel Kaisar; author: Jameel Kaisar):

[mediawiki/extensions/WikimediaEvents@master] Fix: Validation errors for status_code in Probenet reports

https://gerrit.wikimedia.org/r/930286

Change 930286 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] probenet: Fix validation errors for status_code in Probenet reports

https://gerrit.wikimedia.org/r/930286

FYI, still lots of validation errors for network probe. E.g.

https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-default-1-7.0.0-1-2023.06.22?id=dvgM5IgBaSyKCtwGvU49

 '.reports[0].pulses[0].actual_bytes' should be number,
 '.reports[0].pulses[0].status_code' should be number,
 '.reports[0].pulses[0].transfer_bytes' should be number,
...

The patch that was submitted for fixing the 'status_code' was later found out would not fix the error. So, a new patch was submitted. But this will fix validation errors for 'status_code' only and will be merged in wmf.15. I will submit a new patch now which do fix the validation errors for all fields .reports[i].pulses[i].*

Change 932354 had a related patch set uploaded (by Jameel Kaisar; author: Jameel Kaisar):

[mediawiki/extensions/WikimediaEvents@master] probenet: Fix validation errors in Probenet reports

https://gerrit.wikimedia.org/r/932354

In T334417#8957797, @JameelKaisar wrote:

The patch that was submitted for fixing the 'status_code' […]
[…] I will submit a new patch now which do fix the validation errors for all fields .reports[i].pulses[i].*

https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/responseStatus

Chrome: Yes
Firefox: Not implemented
Safari: Not implemented

https://developer.mozilla.org/en-US/docs/Web/API/PerformanceResourceTiming/decodedBodySize

Mobile Safari: iOS 16.4 and later

T334417#8956738 => https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-default-1-7.0.0-1-2023.06.22?id=dvgM5IgBaSyKCtwGvU49

"status_code":null,"transfer_bytes":null,"actual_bytes":null
…
"Mozilla/5.0 (iPhone; CPU iPhone OS 16_1_1 …) Version/16.1 Mobile/… Safari/…"

Mobile Safari 16.1.

These three ResourceTiming/probe_result properties were being standardised in a later iteration of this browser feature, and thus are effectively optional / can be undefined in some browsers. And myObject.somethingElse === undefined, and undefined - undefined, or Math.round(undefined), or Number(undefined), all produce NaN => JSON.stringify(NaN) => null.

Thanks for the detailed explanation. It was very confusing to find some properties being set as null.

Change 932354 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] probenet: Fix validation errors in Probenet reports

https://gerrit.wikimedia.org/r/932354

Change 939239 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/WikimediaEvents@master] probenet: Reduce Probenet#extractProbeData()

https://gerrit.wikimedia.org/r/939239

Change 939239 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] probenet: Reduce Probenet#extractProbeData()

https://gerrit.wikimedia.org/r/939239

The aforementioned validation errors essentially ceased once the above patches were deployed: