Page MenuHomePhabricator

Errors in at reportLoadTiming: Cannot read properties of undefined (reading 'loadEventEnd') / event is undefined / Cannot read property 'loadEventEnd' of undefined
Closed, ResolvedPublicBUG REPORT

Description

Seems to have been introduced in wmf30. This is currently the top client error.
449,290 errors in the last 7 days in Wikimedia Commons only (This is extremely high error rate - 2nd-highest error rate is only 14,681 errors in same period)
Stack traces point to this code:
https://gerrit.wikimedia.org/g/mediawiki/extensions/MediaSearch/+/9c32c81792dbacb89bda6676eb33c570a0090072/resources/components/App.vue#369

stack trace for event is undefined

at reportLoadTiming https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:305:345
at created https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:305:443
at Vt https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:406:797
at Bt https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:406:869
at Bt https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:406:968
at Ir https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:434:869
at resources/lib/vue/vue.js/yi/< https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:467:711
at yi https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:468:188
at resources/lib/vue/vue.js/di/s< https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:465:847
at di https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:465:853
at M https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:450:354
at P https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:450:209
at m https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:447:233
at X https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:457:418
at mount https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:442:678
at resources/lib/vue/vue.js/Xc/t.mount https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:487:870
at resources/init.js/< https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:259:653
at resources/init.js https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%7Cext.centralauth.centralautologin%7Cext.eventLogging%2CwikimediaEvents%7Cmediawiki.base%7Cmediawiki.page.ready&skin=vector&version=5vcka:259:673
at runScript https://commons.wikimedia.org/w/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:11:831
at execute https://commons.wikimedia.org/w/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:13:96
at doPropagation https://commons.wikimedia.org/w/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:5:943

stack trace for Cannot read property 'loadEventEnd' of undefined

at reportLoadTiming  https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:305:389
at Proxy.created  https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:305:443
at Vt  https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:406:797
at Bt  https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:406:867
at Bt  https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:406:968
at Ir  https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:434:867
at https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:467:711
at yi  https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:468:188
at https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:465:845
at di  https://commons.m.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue&skin=minerva&version=7kkzg:465:853

stack trace for undefined is not an object (evaluating 'event.loadEventEnd')

at reportLoadTiming https://commons.m.wikimedia.org/w/load.php:305:388
at created https://commons.m.wikimedia.org/w/load.php:305:459
at Vt https://commons.m.wikimedia.org/w/load.php:406:798
at Bt https://commons.m.wikimedia.org/w/load.php:406:869
at Bt https://commons.m.wikimedia.org/w/load.php:406:970
at Ir https://commons.m.wikimedia.org/w/load.php:434:869
at https://commons.m.wikimedia.org/w/load.php:467:713
at yi https://commons.m.wikimedia.org/w/load.php:468:188
at https://commons.m.wikimedia.org/w/load.php:465:847
at di https://commons.m.wikimedia.org/w/load.php:465:853
at M https://commons.m.wikimedia.org/w/load.php:450:356
at P https://commons.m.wikimedia.org/w/load.php:450:210
at m https://commons.m.wikimedia.org/w/load.php:447:234
at X https://commons.m.wikimedia.org/w/load.php:457:419
at mount https://commons.m.wikimedia.org/w/load.php:442:679
at https://commons.m.wikimedia.org/w/load.php:487:870
at https://commons.m.wikimedia.org/w/load.php:259:658
at resources/init.js https://commons.m.wikimedia.org/w/load.php:259:673
at runScript https://commons.m.wikimedia.org/w/load.php:11:831
at execute https://commons.m.wikimedia.org/w/load.php:13:105
at doPropagation https://commons.m.wikimedia.org/w/load.php:5:943
at https://commons.m.wikimedia.org/w/load.php:22:737

can't access property "loadEventEnd", event is undefined

at reportLoadTiming https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:305:383
at created https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:305:443
at Vt https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:406:797
at Bt https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:406:869
at Bt https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:406:968
at Ir https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:434:869
at resources/lib/vue/vue.js/yi/< https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:467:711
at yi https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:468:188
at resources/lib/vue/vue.js/di/s< https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:465:847
at di https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:465:853
at M https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:450:354
at P https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:450:209
at m https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:447:233
at X https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:457:418
at mount https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:442:678
at resources/lib/vue/vue.js/Xc/t.mount https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:487:870
at resources/init.js/< https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:259:653
at resources/init.js https://commons.wikimedia.org/w/load.php?lang=en&modules=%40wikimedia%2Fcodex%2Cjquery%2Cmediasearch%2Cvue%2Cvuex%7Cext.centralauth.ForeignApi%7Cext.wikimediaEvents.specialPages%7Cmediawiki.ForeignApi%7Cmediawiki.ForeignApi.core&skin=vector&version=ygfdr:259:673
at runScript https://commons.wikimedia.org/w/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:11:831
at execute https://commons.wikimedia.org/w/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:13:96
at doPropagation https://commons.wikimedia.org/w/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector:5:943

Event Timeline

Restricted Application added subscribers: Masumrezarock100, Aklapper. · View Herald Transcript
Jdlrobson triaged this task as Unbreak Now! priority.EditedOct 19 2023, 5:08 AM

Given the high error rate, this could signal a problem in our reporting instrument and is adding significant noise to our logs given the multiple forms of the error.

Change 967427 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/MediaSearch@master] reportLoadTiming: fix typo in var name

https://gerrit.wikimedia.org/r/967427

Change 967427 abandoned by DCausse:

[mediawiki/extensions/MediaSearch@master] reportLoadTiming: fix typo in var name

Reason:

https://gerrit.wikimedia.org/r/967427

Change 967411 had a related patch set uploaded (by DCausse; author: Cparle):

[mediawiki/extensions/MediaSearch@master] Fix typo (undefined event)

https://gerrit.wikimedia.org/r/967411

Change 967411 merged by jenkins-bot:

[mediawiki/extensions/MediaSearch@master] Fix typo (undefined event)

https://gerrit.wikimedia.org/r/967411

Change 967780 had a related patch set uploaded (by Cparle; author: Cparle):

[mediawiki/extensions/MediaSearch@wmf/1.42.0-wmf.1] Fix typo (undefined event)

https://gerrit.wikimedia.org/r/967780

Change 967780 merged by jenkins-bot:

[mediawiki/extensions/MediaSearch@wmf/1.42.0-wmf.1] Fix typo (undefined event)

https://gerrit.wikimedia.org/r/967780

Mentioned in SAL (#wikimedia-operations) [2023-10-24T13:51:28Z] <samtar@deploy2002> Started scap: Backport for [[gerrit:967780|Fix typo (undefined event) (T349271)]]

Mentioned in SAL (#wikimedia-operations) [2023-10-24T13:52:48Z] <samtar@deploy2002> samtar and cparle: Backport for [[gerrit:967780|Fix typo (undefined event) (T349271)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-10-24T14:00:54Z] <samtar@deploy2002> Finished scap: Backport for [[gerrit:967780|Fix typo (undefined event) (T349271)]] (duration: 09m 26s)

Jdlrobson lowered the priority of this task from Unbreak Now! to Needs Triage.Oct 24 2023, 6:33 PM

I can confirm I'm now seeing a clear drop off here: https://logstash.wikimedia.org/goto/d7f24de00fd7ee6cb319c4646a9318f4

Screenshot 2023-10-24 at 11.33.09 AM.png (241×2 px, 31 KB)

Thanks @Jdlrobson ... does that mean we can close this?