Test test_content_format failing due to new member 'application/vnd.php.serialized' on Wikibase repos
Open, HighPublic

Description

The API paraminfo list of values for the edit module has recently changed its list of available content formats. 'application/vnd.php.serialized' now appears as an edit module content format, but only on Wikibase repos as far as the pywikibot CI test suite can determine.

As seen on https://travis-ci.org/jayvdb/pywikibot-core/jobs/114591756

======================================================================
FAIL: Test content format.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/jayvdb/pywikibot-core/tests/paraminfo_tests.py", line 127, in test_content_format
    self._check_param_values(self.site, 'edit', 'contentformat', base)
  File "/home/travis/build/jayvdb/pywikibot-core/tests/paraminfo_tests.py", line 45, in _check_param_values
    self.assertCountEqual(expected, values)
  File "/home/travis/build/jayvdb/pywikibot-core/tests/aspects.py", line 96, in assertCountEqual
    return self.assertItemsEqual(*args, **kwargs)
AssertionError: Sequences differ: [u'ap[14 chars]', u'text/css', u'text/javascript', u'text/pla[15 chars]iki'] != [u'ap[14 chars]', u'application/vnd.php.serialized', u'text/c[50 chars]iki']

First differing element 1:
text/css
application/vnd.php.serialized

Second sequence contains 1 additional elements.
First extra element 5:
text/x-wiki

  [u'application/json',
+  u'application/vnd.php.serialized',
   u'text/css',
   u'text/javascript',
   u'text/plain',
   u'text/x-wiki']

The pywikibot test can add 'application/vnd.php.serialized' to its expected list, but it doesnt always appear.
Ideally we work out why and when it appears, and verify it is indeed intentional that this change has occurred.

jayvdb created this task.Mar 8 2016, 9:52 PM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptMar 8 2016, 9:52 PM
jayvdb updated the task description. (Show Details)Mar 12 2016, 10:52 PM
Restricted Application added a project: Wikidata. · View Herald TranscriptMar 12 2016, 10:57 PM
jayvdb added a subscriber: Bene.

This error did not occur in build 3299(.18 and .20), but does appear in 3200(.18 and .20) . So, the relevant change was deployed on March 9.

On March 9 there was commit ccbbd68701e43 (T129349 is absolutely empty, explaining nothing except a parent task). It is about content models.

On March 2 there is 4d613d2f3be4 , which looks closer to the type of change which might have this result.

Bene added a comment.EditedMar 13 2016, 6:27 PM

By grepping in Wikibase I can only find this file but the relevant code was touched last on Jun 17 2015.

The issue has clearly nothing to do with the commits you linked as they are only internal refactoring without any changes to api output formats.

Maybe there was a change in core or a config change enabling this output format?

@Bene , MediaWiki core change is unlikely, as it only occurs on Wikidata , but I will add MediaWiki-API just in case (and knowing they can usually help with oddities like this) . A config is possible. My (very very uninformed) guess is that some of your Wikibase internal refactoring changes have inadvertently caused MediaWiki to believe this content format is 'needed' by the API.

Anomie added a subscriber: Anomie.Mar 14 2016, 2:14 PM

The thing being complained about here is the list of accepted values to the contentformats parameter to action=edit. This list of possible values is fetched from ContentHandler::getAllContentFormats(), which queries every handler in $wgContentHandlers for ->getSupportedFormats(), and Wikibase's handlers for "wikibase-item" (Wikibase\Repo\Content\ItemHandler) and "wikibase-property" (Wikibase\Repo\Content\PropertyHandler) are currently returning "application/vnd.php.serialized" (the value of the constant CONTENT_FORMAT_SERIALIZED) as one of their supported formats.

I have no idea why that sequence of events might not have been happening in the same way before March 9.

jayvdb triaged this task as High priority.Mar 19 2016, 12:20 AM

Our test suite has a failing test because of this. Unless it was intentionally exposed, and it seems that it wasnt, please find the cause and remove it.

You're assuming that it's incorrect for it to be exposed. That is far from evident here, it could well be that the bug was that it wasn't being listed before when it should have been, and something recently accidentally fixed that.

I'd be happy either way; but would like to know promptly either way. If it is correct for it to be exposed, then we can update our test method. We also need to document how/when to use application/vnd.php.serialized with action=edit. It isnt currently listed on https://www.mediawiki.org/wiki/API:Edit

https://phabricator.wikimedia.org/diffusion/MW/browse/master/docs/contenthandler.txt also needs to be updated, as it currently indicates it is reserved for future use:

* application/vnd.php.serialized - for future use with the api and for extensions
* application/json - for future use with the api, and for use by extensions
* application/xml - for future use with the api, and for use by extensions
Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.May 3 2016, 10:35 AM

Change 342162 had a related patch set uploaded (by Dalba):
[pywikibot/core] Add 'application/vnd.php.serialized' to expected content formats of DataSite

https://gerrit.wikimedia.org/r/342162

Change 342162 merged by jenkins-bot:
[pywikibot/core] Add 'application/vnd.php.serialized' to expected content formats of DataSite

https://gerrit.wikimedia.org/r/342162