Mentioned in SAL (#wikimedia-operations) [2017-06-22T19:52:27Z] <twentyafterfour> the train is currently blocked by https://phabricator.wikimedia.org/T168681

@aude or @Addshore: Any clue what might be causing this? I'm not sure how vendor'd code works in the wikidata extension. This error started showing up suddenly after the most recent SWAT even though no code was deployed which should have affected this.

It occurs at API level (so could explain why it disappeared, then came back) in UnboundedQuantityValue class:

#0 /srv/mediawiki/php-1.30.0-wmf.6/extensions/Wikidata/vendor/data-values/number/src/DataValues/UnboundedQuantityValue.php(277): MWExceptionHandler::handleError(integer, string, string, integer, array, array)
#1 /srv/mediawiki/php-1.30.0-wmf.6/extensions/Wikidata/vendor/data-values/serialization/src/Deserializers/DataValueDeserializer.php(141): DataValues\UnboundedQuantityValue::newFromArray(string)
#2 /srv/mediawiki/php-1.30.0-wmf.6/extensions/Wikidata/vendor/data-values/serialization/src/Deserializers/DataValueDeserializer.php(97): DataValues\Deserializers\DataValueDeserializer->getDeserialization()

A deserialized value is expected to be an array, but is a string. The UnboundedQuantityValue code didn't change these last two months.

That's what I don't understand, the code didn't change... So you think this is just invalid data passed to the API and the timing (which coincided with SWAT) was just a coincidence?

It seems to be possible to produce this error via the API, at least, using api.php?action=wbformatvalue&format=json&datavalue=%7B%20%22type%22%3A%20%22quantity%22%2C%20%22value%22%3A%20%2210%22%20%7D (i. e., wbformatvalue with datavalue { "type": "quantity", "value": "10" }). I tried it on my local installation and got the same error message – on Wikidata, I get HTTP 500 but there’s no error message or stack trace in the response, but I assume that’s normal.

With this API request, the stack trace I get is https://gist.github.com/lucaswerkmeister/fa0a142dfcc3360daa72c0c1b4e9ba3c – does that mostly match the stack trace you saw, @Dereckson?

In T168681#3373536, @Lucas_Werkmeister_WMDE wrote:

With this API request, the stack trace I get is https://gist.github.com/lucaswerkmeister/fa0a142dfcc3360daa72c0c1b4e9ba3c

which is:

#0 /var/www/html/wiki1/extensions/WikibaseQualityConstraints/vendor/data-values/serialization/src/Deserializers/DataValueDeserializer.php(141): DataValues\UnboundedQuantityValue::newFromArray(string)<br />
#1 /var/www/html/wiki1/extensions/WikibaseQualityConstraints/vendor/data-values/serialization/src/Deserializers/DataValueDeserializer.php(97): DataValues\Deserializers\DataValueDeserializer-&gt;getDeserialization()<br />
#2 /var/www/html/wiki1/extensions/Wikibase/lib/includes/DataValueFactory.php(42): DataValues\Deserializers\DataValueDeserializer-&gt;deserialize(array)<br />
#3 /var/www/html/wiki1/extensions/Wikibase/lib/includes/DataValueFactory.php(73): DataValues\DataValueFactory-&gt;newDataValue(string, string)<br />
#4 /var/www/html/wiki1/extensions/Wikibase/repo/includes/Api/FormatSnakValue.php(182): DataValues\DataValueFactory-&gt;newFromArray(array)<br />
#5 /var/www/html/wiki1/extensions/Wikibase/repo/includes/Api/FormatSnakValue.php(89): Wikibase\Repo\Api\FormatSnakValue-&gt;decodeDataValue(string)<br />
#6 /var/www/html/wiki1/includes/api/ApiMain.php(1578): Wikibase\Repo\Api\FormatSnakValue-&gt;execute()<br />
#7 /var/www/html/wiki1/includes/api/ApiMain.php(545): ApiMain-&gt;executeAction()<br />
#8 /var/www/html/wiki1/includes/api/ApiMain.php(516): ApiMain-&gt;executeActionWithErrorHandling()<br />
#9 /var/www/html/wiki1/api.php(83): ApiMain-&gt;execute()<br />
#10 {main}<br />

What I want to know is why this just started happening. If it's not really related to today's SWAT then perhaps it shouldn't block the train.

haven't investigated yet but my guess is it's a bot misbehaving

PS we prepared a bump (to wmf/1.30.0-wmf.6) for Wikidata extensions on Tuesday but forgot to merge the bump in release tools in time for the branch cut.

so, Wikidata extensions are still on older version (wmf/1.30.0-wmf.4) which has been deployed for 2 weeks. So I don't believe this is related to deployment.

(nonetheless can take a look)

the servers that the errors occurred on are all api servers, so it's something hitting the api

After looking into this, I still think this was caused by a bot misbehaving. No relevant code has changed/deployed recently.

The error stopped (and briefly occurred again around 2:28 UTC).

We can try to have code changes out next week (in regular deployment / wmf.7) that handle such bad input better. I don't think this needs to block wmf.6 deployment also don't think swat backports are necessary if this error only happens sporadically.

Removing this as a deployment blocker per the investigation. Thanks all.

Change 361406 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Update several DataValues libraries to current bug fix releases

https://gerrit.wikimedia.org/r/361406

Change 361406 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Update several DataValues libraries to current bug fix releases

https://gerrit.wikimedia.org/r/361406