Change Details

On October 5th 2020, an edit to pt:MediaWiki:Common.js led to a spike in client side errors. In the space of an hour 20,000 errors were logged, compared with 15,000 in the 12hrs across all wikis that preceded that [1]. This code appears to have been run for any anonymous user. Had this occurred on English Wikipedia where volume is considerably higher it would have likely brought down the system and is one of the big reasons why we haven't enabled on English Wikipedia. {F32374697} In an ideal world, the editor of the site JS would have had access to logstash (or some anonymized equivalent) and have been able to check their change - in 5 minutes it would have been clear there was a problem. However, the error was left for 30 minutes, until luckily I chanced upon it in an open tab set to auto-refresh and was able to [[ https://www.mediawiki.org/wiki/Reading/Web/Notable_incidents#October | rush a fix myself ]]. I won't always catch such errors. The stack trace gave no indication for where the error originated so these kind of errors cannot be filtered easily. The only clue was "site" module in the name of the file_uri however this applies also for errors we care about. See T262493 for more details on the larger problem here. ``` at Anonymous function Unknown script code:2:535 at mightThrow URL1:50:132 at Anonymous function URL1:50:808 `` [1] https://www.mediawiki.org/wiki/Reading/Web/Notable_incidents#October

The purpose of this task is to identify a way to exclude or limit error reportages when such breakages occur and provide better processes to help editors identify and address such problems. On October 5th 2020, an edit to pt:MediaWiki:Common.js led to a spike in client side errors. In the space of an hour 20,000 errors were logged, compared with 15,000 in the 12hrs across all wikis that preceded that [1]. This code appears to have been run for any anonymous user. Had this occurred on English Wikipedia where volume is considerably higher it would have likely brought down the system and is one of the big reasons why we haven't enabled on English Wikipedia. {F32374697} In an ideal world, the editor of the site JS would have had access to logstash (or some anonymized equivalent) and have been able to check their change - in 5 minutes it would have been clear there was a problem. However, the error was left for 30 minutes, until luckily I chanced upon it in an open tab set to auto-refresh and was able to [[ https://www.mediawiki.org/wiki/Reading/Web/Notable_incidents#October | rush a fix myself ]]. I won't always catch such errors. The stack trace gave no indication for where the error originated so these kind of errors cannot be filtered easily. The only clue was "site" module in the name of the file_uri however this applies also for errors we care about. See T262493 for more details on the larger problem here. ``` at Anonymous function Unknown script code:2:535 at mightThrow URL1:50:132 at Anonymous function URL1:50:808 `` [1] https://www.mediawiki.org/wiki/Reading/Web/Notable_incidents#October

The purpose of this task is to identify a way to exclude or limit error reportages when such breakages occur and provide better processes to help editors identify and address such problems. On October 5th 2020, an edit to pt:MediaWiki:Common.js led to a spike in client side errors. In the space of an hour 20,000 errors were logged, compared with 15,000 in the 12hrs across all wikis that preceded that [1]. This code appears to have been run for any anonymous user. Had this occurred on English Wikipedia where volume is considerably higher it would have likely brought down the system and is one of the big reasons why we haven't enabled on English Wikipedia. {F32374697} In an ideal world, the editor of the site JS would have had access to logstash (or some anonymized equivalent) and have been able to check their change - in 5 minutes it would have been clear there was a problem. However, the error was left for 30 minutes, until luckily I chanced upon it in an open tab set to auto-refresh and was able to [[ https://www.mediawiki.org/wiki/Reading/Web/Notable_incidents#October | rush a fix myself ]]. I won't always catch such errors. The stack trace gave no indication for where the error originated so these kind of errors cannot be filtered easily. The only clue was "site" module in the name of the file_uri however this applies also for errors we care about. See T262493 for more details on the larger problem here. ``` at Anonymous function Unknown script code:2:535 at mightThrow URL1:50:132 at Anonymous function URL1:50:808 `` [1] https://www.mediawiki.org/wiki/Reading/Web/Notable_incidents#October