Page MenuHomePhabricator

A malformed(?) production error corrupted the error channels, with the message field being over-written with `class@anonymous`
Closed, ResolvedPublic

Description

Follow-up from T245062.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 14 2020, 7:32 PM

Source: https://gerrit.wikimedia.org/g/mediawiki/extensions/Wikibase/+/b5763d548e1b843d24ff085b03b07767b8272c23/lib/includes/SimpleCacheWithBagOStuff.php

		if ( !preg_match( self::KEY_REGEX, $key ) ) {
			throw $this->invalidArgument( "Cache key contains characters that are not allowed: `{$key}`" ); # line 266
		}
	}
	private function invalidArgument( $message ) {
		return new class( $message ) extends \InvalidArgumentException implements InvalidArgumentException {
		};
	}

JSON record from Logstash:

{
    "exception": {
      "trace": "#0 /srv/mediawiki/php-1.35.0-wmf.19/extensions/Wikibase/lib/includes/SimpleCacheWithBagOStuff.php(266): Wikibase\\Lib\\SimpleCacheWithBagOStuff->invalidArgument(string)\n#1 /srv/mediawiki/php-1.35.0-wmf.19/extensions/Wikibase/lib/includes/SimpleCacheWithBagOStuff.php(233): Wikibase\\Lib\\SimpleCacheWithBagOStuff->assertKeyIsValid(string)\n#2 /srv/mediawiki/php-1.35.0-wmf.19/extensions/Wikibase/lib/includes/Store/UncachedTermsPrefetcher.php(76): Wikibase\\Lib\\SimpleCacheWithBagOStuff->has(string)\n#3 /srv/mediawiki/php-1.35.0-wmf.19/extensions/Wikibase/lib/includes/Store/UncachedTermsPrefetcher.php(41): ………\n#53 {main}",
      "code": 0,
      "file": "/srv/mediawiki/php-1.35.0-wmf.19/extensions/Wikibase/lib/includes/SimpleCacheWithBagOStuff.php:271",
      "message": "Cache key contains characters that are not allowed: `P31_1111607050_⧼lang⧽_label`",
      "class": "class@anonymous"
    },
    "server": "commons.wikimedia.org",
    "phpversion": "7.2.26-1+0~20191218.33+debian9~1.gbpb5a340+wmf1",
    "wiki": "commonswiki",
    "channel": "exception",
    "exception_id": "XkV86QpAICsAACWu934AAAAB",
    "program": "mediawiki",
    "type": "mediawiki",
    "message_checksum": "7e484ce0e90854049eec05efe58a1e41",
    "caught_by": "mwe_handler",
    "exception_url": "/wiki/Category:  …  ?uselang=%E2%A7%BCLang%E2%A7%BD",
    "http_method": "GET",
    "host": "mw1322",
    "@version": 1,
    "shard": "s4",
    "timestamp": "2020-02-13T16:44:25+00:00",
    "severity": "err",
    "unique_id": "XkV86QpAICsAACWu934AAAAB",
    "level": "ERROR",
    "ip": "10.64.32.43",
    "mwversion": "1.35.0-wmf.19",
    "logsource": "mw1322",
    "message": "[XkV86QpAICsAACWu934AAAAB] /wiki/Category:   …  ?uselang=%E2%A7%BCLang%E2%A7%BD   class@anonymous",
    "normalized_message": "[{exception_id}] {exception_url}   class@anonymous",
    "url": "/wiki/Category:  …  ?uselang=%E2%A7%BCLang%E2%A7%BD",
    "reqId": "XkV86QpAICsAACWu934AAAAB",
    "tags": [
      "input-kafka-rsyslog-udp-localhost",
      "rsyslog-udp-localhost",
      "kafka",
      "es"
    ],
    "referrer": null,
    "@timestamp": "2020-02-13T16:44:25.843Z",
    "facility": "user"
  }

Why exactly is this throwing an anonymous class instead of just an InvalidArgumentException?

Change 576441 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/Wikibase@master] SimpleCacheWithBagOStuff: Throw InvalidArgumentException directly

https://gerrit.wikimedia.org/r/576441

Addshore added a subscriber: Addshore.

We probably just want to throw a real implementation of said interface..

Addshore triaged this task as Medium priority.Mar 4 2020, 1:53 PM
Addshore added subscribers: LarsWirzenius, Tarrow.

I think this can be merged into T245295.

In my opinion this ticket shows a symptom of at least 3 separate problems:

  • T245295 : Production logging of this is broken / unclear because of the error class that is thrown
  • T245396: The Wikibase implementation of a SimpleCache backed by a BagOStuff doesn't take advantage of the methods provided by BagOStuff to reduce the incidence of attempts to use bad keys
  • T247057: A Wikibase caching lookup is wrongly trying to do lookups for languages that are not valid Term Languages
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptMar 6 2020, 11:21 AM
Maintenance_bot moved this task from Incoming to In progress on the User-Ladsgroup board.

Change 576441 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] SimpleCacheWithBagOStuff: Throw CacheInvalidArgumentException

https://gerrit.wikimedia.org/r/576441

Jdforrester-WMF closed this task as Resolved.Mar 12 2020, 1:51 AM

This appears to now be fixed. Thanks!