Page MenuHomePhabricator

Update pywikibot queries for MCR
Closed, ResolvedPublic

Description

MediaWiki 1.32/wmf.15 updated API query modules for MCR.

See rMWfb1b454e6a6f: Merge "API: Update query modules for MCR" / T200568: Make API query modules aware of MCR:

  • action=query&prop=revisions now takes a 'rvslots' parameter to indicate the multi-content revision slots for which content should be returned. It also has a new rvprop, 'roles', to indicate which roles have slots. A deprecation warning will be issued if rvprop=content or rvprop=contentmodel are used without rvslots.
  • The rvcontentformat parameter to action=query&prop=revisions has been deprecated. Clients should be prepared to deal with the default format for relevant models.
  • Use of the deprecated parameters rvexpandtemplates, rvgeneratexml, rvparse, rvdiffto, rvdifftotext, rvdifftotextpst, rvcontentformat, or the deprecated rvprop=parsetree is forbidden with the new 'rvslots' parameter.
  • action=query&prop=deletedrevisions, action=query&list=allrevisions, and action=query&list=alldeletedrevisions are changed similarly to &prop=revisions (see the three previous items).

Today the below started appearing in my logs for Commons.

2018-08-02 01:04:36             api.py, 1900 in   _handle_warnings: WARNING  API warning (main): Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]] to see usage of deprecated features by your application.
2018-08-02 01:04:36             api.py, 1900 in   _handle_warnings: WARNING  API warning (revisions): Because "rvslots" was not specified, a legacy format has been used for the output. This format is deprecated, and in the future the new format will always be used.

Event Timeline

The same warnings just appeared on frwiki.

Dalba triaged this task as High priority.EditedAug 3 2018, 2:47 AM
Dalba added a project: Pywikibot-tests.
Dalba subscribed.

Some Travis-CI jobs have strated to fail because of the new deprecation warning:
https://travis-ci.org/wikimedia/pywikibot/jobs/411423632#L1953

=================================== FAILURES ===================================
_____________________ TestScriptSimulate.test_editarticle ______________________
self = <tests.script_tests.TestScriptSimulate testMethod=test_editarticle>
    def testScript(self):
        global_args = 'Global arguments available for all'
    
        cmd = [script_name]
    
        if args:
            cmd += args
    
        data_in = script_input.get(script_name)
    
        timeout = 0
        if is_autorun:
            timeout = 5
    
        if self._results and script_name in self._results:
            error = self._results[script_name]
            if isinstance(error, StringTypes):
                stdout = None
            else:
                stdout, error = error
        else:
            stdout = None
            error = None
    
        test_overrides = {}
        if not hasattr(self, 'net') or not self.net:
            test_overrides['pywikibot.Site'] = 'None'
    
        result = execute_pwb(cmd, data_in, timeout=timeout,
                             error=error, overrides=test_overrides)
    
        stderr = result['stderr'].splitlines()
        stderr_sleep = [l for l in stderr
                        if l.startswith('Sleeping for ')]
        stderr_other = [l for l in stderr
                        if not l.startswith('Sleeping for ')]
        if stderr_sleep:
            unittest_print('\n'.join(stderr_sleep))
    
        if result['exit_code'] == -9:
            unittest_print(' killed', end='  ')
    
        if error:
            self.assertIn(error, result['stderr'])
    
            exit_codes = [0, 1, 2, -9]
        elif not is_autorun:
            if stderr_other == []:
                stderr_other = None
            if stderr_other is not None:
                self.assertIn('Use -help for further information.',
                              stderr_other)
                self.assertNotIn('-help', args)
            else:
                self.assertIn(global_args, result['stdout'])
    
            exit_codes = [0]
        else:
            # auto-run
            exit_codes = [0, -9]
    
            if (not result['stdout'] and not result['stderr']):
                unittest_print(' auto-run script unresponsive after '
                               '%d seconds' % timeout, end=' ')
            elif 'SIMULATION: edit action blocked' in result['stderr']:
                unittest_print(' auto-run script simulated edit '
                               'blocked', end='  ')
            else:
                unittest_print(
                    ' auto-run script stderr within %d seconds: %r'
                    % (timeout, result['stderr']), end='  ')
    
        self.assertNotIn('Traceback (most recent call last)',
                         result['stderr'])
>       self.assertNotIn('deprecated', result['stderr'].lower())
E       AssertionError: 'deprecated' unexpectedly found in 'page to edit: warning: api warning (main): subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of api deprecations and breaking changes. use [[special:apifeatureusage]] to see usage of deprecated features by your application.\nwarning: api warning (revisions): because "rvslots" was not specified, a legacy format has been used for the output. this format is deprecated, and in the future the new format will always be used.\nnothing changed\n'

Change 450460 had a related patch set uploaded (by Dalba; owner: dalba):
[pywikibot/core@master] Use 'rvslots' when fetching revision on MW 1.32+

https://gerrit.wikimedia.org/r/450460

Restricted Application added a subscriber: alaa. · View Herald TranscriptAug 5 2018, 7:03 PM

Change 450460 merged by jenkins-bot:
[pywikibot/core@master] Use 'rvslots' when fetching revisions on MW 1.32+

https://gerrit.wikimedia.org/r/450460

Framawiki assigned this task to Dalba.

Since the patch was merged and as I know nothing else is needed today, let's close this task.

Xqt subscribed.

Get this message again for some scripts

Get this message again for some scripts

Which scripts? Could you provide sample code, please?

C:\pwb\GIT\core>pwb.py makecat -simulate Hydraulik
Retrieving 5 pages from wikipedia:de.
WARNING: API warning (main): Subscribe to the mediawiki-api-announce mailing lis
t at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for n
otice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]]
to see usage of deprecated features by your application.
WARNING: API warning (revisions): Because "rvslots" was not specified, a legacy
format has been used for the output. This format is deprecated, and in the futur
e the new format will always be used.
WARNING: API warning (main): Subscribe to the mediawiki-api-announce mailing lis
t at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for n
otice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]]
to see usage of deprecated features by your application.
WARNING: API warning (revisions): Because "rvslots" was not specified, a legacy
format has been used for the output. This format is deprecated, and in the futur
e the new format will always be used.

Digging deeper in this matter: The Problem occures with the "content" parameter for Category.articles():

>>> import pwb, pywikibot as py
>>> s = py.Site()
>>> c = py.Category(s, 'Hydraulik')
>>> a = list(c.articles())
>>> a = list(c.articles(content=True))
WARNING: API warning (main): Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]] to see usage of deprecated features by your application.
WARNING: API warning (revisions): Because "rvslots" was not specified, a legacy format has been used for the output. This format is deprecated, and in the future the new format will always be used.
WARNING: API warning (main): Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]] to see usage of deprecated features by your application.
WARNING: API warning (revisions): Because "rvslots" was not specified, a legacy format has been used for the output. This format is deprecated, and in the future the new format will always be used.
>>>

Change 462148 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] Use 'rvslots' when fetching categorymember() content on MW 1.32+

https://gerrit.wikimedia.org/r/462148

Change 462148 merged by jenkins-bot:
[pywikibot/core@master] Use 'rvslots' when fetching categorymember() content on MW 1.32+

https://gerrit.wikimedia.org/r/462148

Change 462149 had a related patch set uploaded (by Dalba; owner: dalba):
[pywikibot/core@master] pywikibot.data.api.PageGenerator: Add rvslots=* on MW 1.32+

https://gerrit.wikimedia.org/r/462149

Change 462149 merged by jenkins-bot:
[pywikibot/core@master] api.QueryGenerator: Handle slots during initialization

https://gerrit.wikimedia.org/r/462149