Page MenuHomePhabricator

TestSiteGenerators.test_unconnected and test_unconnected_with_repo are failing with UnknownExtension error
Closed, ResolvedPublic

Description

https://travis-ci.org/wikimedia/pywikibot/jobs/398125190#L3669:

=================================== FAILURES ===================================
___________ TestUnconnectedPageGenerator.test_unconnected_with_repo ____________
self = <tests.pagegenerators_tests.TestUnconnectedPageGenerator testMethod=test_unconnected_with_repo>
    def test_unconnected_with_repo(self):
        """Test UnconnectedPageGenerator."""
        if not self.site.data_repository():
            raise unittest.SkipTest('Site is not using a Wikibase repository')
        with suppress_warnings(
                'pywikibot.pagegenerators.UnconnectedPageGenerator is '
                'deprecated', DeprecationWarning):
>           upgen = pagegenerators.UnconnectedPageGenerator(self.site, 3)
tests/pagegenerators_tests.py:1501: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pywikibot/tools/__init__.py:1324: in wrapper
    return obj(*args, **kwargs)
pywikibot/tools/__init__.py:1446: in wrapper
    return obj(*__args, **__kw)
pywikibot/pagegenerators.py:1452: in UnconnectedPageGenerator
    return site.unconnected_pages(total=total)
pywikibot/tools/__init__.py:1446: in wrapper
    return obj(*__args, **__kw)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = APISite("en", "wikipedia"), args = (), kwargs = {'total': 3}
    def callee(self, *args, **kwargs):
        if not self.has_extension(extension):
            raise UnknownExtension(
                'Method "%s" is not implemented without the extension %s'
>               % (fn.__name__, extension))
E           UnknownExtension: Method "unconnected_pages" is not implemented without the extension Wikibase Client
pywikibot/site.py:1363: UnknownExtension
_____________________ TestSiteGenerators.test_unconnected ______________________
self = <tests.site_tests.TestSiteGenerators testMethod=test_unconnected>
    def test_unconnected(self):
        """Test site.unconnected_pages method."""
        if not self.site.data_repository():
            raise unittest.SkipTest('Site is not using a Wikibase repository')
>       upgen = self.site.unconnected_pages(total=3)
tests/site_tests.py:1095: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pywikibot/tools/__init__.py:1446: in wrapper
    return obj(*__args, **__kw)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = APISite("en", "wikipedia"), args = (), kwargs = {'total': 3}
    def callee(self, *args, **kwargs):
        if not self.has_extension(extension):
            raise UnknownExtension(
                'Method "%s" is not implemented without the extension %s'
>               % (fn.__name__, extension))
E           UnknownExtension: Method "unconnected_pages" is not implemented without the extension Wikibase Client
pywikibot/site.py:1363: UnknownExtension

Event Timeline

Dalba created this task.Jun 29 2018, 7:41 AM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJun 29 2018, 7:41 AM
Dalba triaged this task as High priority.Jun 29 2018, 7:41 AM
Dalba claimed this task.Jun 29 2018, 8:14 AM

Change 443041 had a related patch set uploaded (by Dalba; owner: dalba):
[pywikibot/core@master] APISite.unconnected_pages: Require WikibaseClient instead of "Wikibase Client"

https://gerrit.wikimedia.org/r/443041

Change 443041 merged by jenkins-bot:
[pywikibot/core@master] APISite.unconnected_pages: Require WikibaseClient instead of "Wikibase Client"

https://gerrit.wikimedia.org/r/443041

Dalba closed this task as Resolved.Jun 29 2018, 11:14 AM
This comment was removed by Dalba.
Dvorapa reopened this task as Open.Jul 1 2018, 12:29 PM
Dvorapa removed a project: Patch-For-Review.
Dvorapa added a subscriber: Dvorapa.

This issue was fixed on Travis but the fix broke AppVeyor :/

Dalba added a comment.Jul 2 2018, 2:23 AM

My best guess is that AppVeyor account is configured to retain cache among builds and therefore it's using the old API results...

Change 443343 had a related patch set uploaded (by Dalba; owner: dalba):
[pywikibot/core@master] Disable the need_extension('WikibaseClient') decorator on unconnected_pages

https://gerrit.wikimedia.org/r/443343

Change 443343 merged by jenkins-bot:
[pywikibot/core@master] Disable the need_extension('WikibaseClient') decorator on unconnected_pages

https://gerrit.wikimedia.org/r/443343

Change 443344 had a related patch set uploaded (by Dvorapa; owner: Dvorapa):
[pywikibot/core@master] Revert "Disable the need_extension('WikibaseClient') decorator on unconnected_pages"

https://gerrit.wikimedia.org/r/443344

Dalba changed the task status from Open to Stalled.Jul 2 2018, 3:26 AM
Dalba lowered the priority of this task from High to Low.
Dalba moved this task from Backlog to Waiting on other changes on the Pywikibot board.
Dvorapa moved this task from Backlog to Test failures on the Pywikibot-tests board.Jul 3 2018, 4:58 PM

Let's try it now

Change 443344 merged by jenkins-bot:
[pywikibot/core@master] Revert "Disable the need_extension('WikibaseClient') decorator on unconnected_pages"

https://gerrit.wikimedia.org/r/443344

AppVeyor still failing. Are we sure this is caused only by cache? I think there is some other mistake there...

Xqt changed the task status from Stalled to Open.Sep 1 2018, 4:56 PM
Dalba raised the priority of this task from Low to High.Sep 2 2018, 6:48 AM
Dalba moved this task from Waiting on other changes to Backlog on the Pywikibot board.

I ran some test builds and it turned out that if only the specific failing tests are run, then they pass OK, but when running the whole test suite they fail. When the tests are failing the extension name is Wikibase Client.

I printed the value of siteinfo['extensions'] right before the failing part and saw this entry for wikibase client:

[00:04:09]  {u'author': u'The Wikidata team',
[00:04:09]   u'credits': u'/wiki/Special:Version/Credits/Wikibase_Client',
[00:04:09]   u'descriptionmsg': u'wikibase-client-desc',
[00:04:09]   u'license': u'/wiki/Special:Version/License/Wikibase_Client',
[00:04:09]   u'license-name': u'GPL-2.0+',
[00:04:09]   u'name': u'Wikibase Client',
[00:04:09]   u'type': u'wikibase',
[00:04:09]   u'url': u'https://www.mediawiki.org/wiki/Extension:Wikibase_Client',
[00:04:09]   u'vcs-date': u'2018-01-03T16:59:50Z',
[00:04:09]   u'vcs-system': u'git',
[00:04:09]   u'vcs-url': u'https://phabricator.wikimedia.org/r/revision/mediawiki/extensions/Wikibase;61f3d290375663c6dcd4aee6ee596e888bd25a04',
[00:04:09]   u'vcs-version': u'61f3d290375663c6dcd4aee6ee596e888bd25a04'},

Notice the 2018-01-03T16:59:50Z date tag. It's very old. Where is it comming from?: The .yml files that store the responses for vcrpy (T184081).

This is what's happening: Sometime during the build, a vcrpy test is ran, that causes the Siteinfo cache to be loaded from the frozen response in the yml files. This old response will be stored in Siteinfo's cache. Other tests will start using the old response...

I'm thinking that it would be much easier to just remove vcrpy tests and decline T184081. I've had several issues with it so far and I don't believe it's worth the trouble.

Change 457082 had a related patch set uploaded (by Dalba; owner: dalba):
[pywikibot/core@master] Revert "Setup VCR for dry tests"

https://gerrit.wikimedia.org/r/457082

Change 457082 merged by jenkins-bot:
[pywikibot/core@master] Revert "Setup VCR for dry tests"

https://gerrit.wikimedia.org/r/457082