TestFactoryGeneratorWikibase.test_searchitem_language is failing in Travis
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	rafidaslam
	Jan 3 2018, 10:57 PM

Description

Most of the jobs in my pywikibot Travis are failing on the master: https://travis-ci.org/refeed/pywikibot/builds/324761525 . All of them is caused by TestFactoryGeneratorWikibase.test_searchitem_language is failing..

=================================== FAILURES ===================================
____________ TestFactoryGeneratorWikibase.test_searchitem_language _____________
self = <tests.pagegenerators_tests.TestFactoryGeneratorWikibase testMethod=test_searchitem_language>
    def test_searchitem_language(self):
        """Test -searchitem with custom language specified."""
        gf = pagegenerators.GeneratorFactory(site=self.site)
        gf.handleArg('-searchitem:pl:abc')
        gf.handleArg('-limit:1')
        gen = gf.getCombinedGenerator()
        self.assertIsNotNone(gen)
        # ABC
        page1 = next(gen)
>       self.assertEqual(page1.title(), 'Q287078')
E       AssertionError: u'Q169889' != u'Q287078'
E       - Q169889
E       + Q287078

Details

	Subject	Repo	Branch	Lines +/-
	pagegenerators_tests: Update test_searchitem_language	pywikibot/core	master	+3 -3

Customize query in gerrit

Related Objects

Mentioned Here: T184081: Convert pywikibot wikibase tests to dry tests by using vcrpy

Event Timeline

rafidaslam created this task.Jan 3 2018, 10:57 PM

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJan 3 2018, 10:57 PM

A possible long term solution for this is https://phabricator.wikimedia.org/T184081

In https://travis-ci.org/refeed/pywikibot/jobs/324761540

____________ TestFactoryGeneratorWikibase.test_searchitem_language _____________
self = <tests.pagegenerators_tests.TestFactoryGeneratorWikibase testMethod=test_searchitem_language>
    def test_searchitem_language(self):
        """Test -searchitem with custom language specified."""
        gf = pagegenerators.GeneratorFactory(site=self.site)
        gf.handleArg('-searchitem:pl:abc')
        gf.handleArg('-limit:1')
        gen = gf.getCombinedGenerator()
        self.assertIsNotNone(gen)
        # ABC
        page1 = next(gen)
>       self.assertEqual(page1.title(), 'Q287078')
E       AssertionError: u'Q169889' != u'Q287078'
E       - Q169889
E       + Q287078
tests/pagegenerators_tests.py:1229: AssertionError
------------------------------ Captured log call -------------------------------
connectionpool.py          396 DEBUG    https://www.wikidata.org:443 "GET /w/api.php?search=abc&maxlag=5&language=pl&format=json&continue=0&action=wbsearchentities HTTP/1.1" 200 665

This is what I got from performing a GET request to that api endpoint:

{"searchinfo":{"search":"abc"},"search":[{"repository":"","id":"Q169889","concepturi":"http://www.wikidata.org/entity/Q169889","title":"Q169889","pageid":170288,"url":"//www.wikidata.org/wiki/Q169889","label":"American Broadcasting Company","description":"American broadcast television network owned by The Walt Disney Company through Disney-ABC Television Group","match":{"type":"alias","language":"pl","text":"ABC Network"},"aliases":["ABC Network"]},{"repository":"","id":"Q781365","concepturi":"http://www.wikidata.org/entity/Q781365","title":"Q781365","pageid":734387,"url":"//www.wikidata.org/wiki/Q781365","label":"Australian Broadcasting Corporation","description":"Australia's state-owned and funded national public radio and TV-broadcaster","match":{"type":"alias","language":"en","text":"ABC"},"aliases":["ABC"]},{"repository":"","id":"Q287078","concepturi":"http://www.wikidata.org/entity/Q287078","title":"Q287078","pageid":277521,"url":"//www.wikidata.org/wiki/Q287078","label":"ABC","description":"English New Romantic band","match":{"type":"label","language":"pl","text":"ABC"}},{"repository":"","id":"Q287076","concepturi":"http://www.wikidata.org/entity/Q287076","title":"Q287076","pageid":277519,"url":"//www.wikidata.org/wiki/Q287076","label":"ABC","description":"Spanish newspaper","match":{"type":"label","language":"pl","text":"ABC (dziennik hiszpa\u0144ski)"},"aliases":["ABC (dziennik hiszpa\u0144ski)"]},{"repository":"","id":"Q1057802","concepturi":"http://www.wikidata.org/entity/Q1057802","title":"Q1057802","pageid":1006628,"url":"//www.wikidata.org/wiki/Q1057802","label":"ABC","description":"programming language","match":{"type":"label","language":"pl","text":"ABC"}},{"repository":"","id":"Q3600480","concepturi":"http://www.wikidata.org/entity/Q3600480","title":"Q3600480","pageid":3428721,"url":"//www.wikidata.org/wiki/Q3600480","label":"ABC","description":"first Australian public television network","match":{"type":"label","language":"en","text":"ABC"}},{"repository":"","id":"Q265653","concepturi":"http://www.wikidata.org/entity/Q265653","title":"Q265653","pageid":257480,"url":"//www.wikidata.org/wiki/Q265653","label":"CBRN defense","description":"CBRN","match":{"type":"label","language":"pl","text":"ABC"},"aliases":["ABC"]}],"search-continue":7,"success":1}

I think the order of list members in that response has changed.. that's why we got Q169889 in Travis. But mysteriously the test passed in my local environment..

Try cleaning your cache (see maintenance script) or just delete tests/apicache*, to check whether you can reproduce locally.

If the order has changed in the Wikidata search results, and this happens consistently, we only need to update the unit test. This would be made unnecessary with T184081 , but using dry data also means we don't notice changes like this, and sometimes these changes are critical problems and pywikibot needs to be updated immediately otherwise user scripts will break.

@jayvdb Ah, yes, I can reproduce it locally by deleting the apicache directory

Here's the log:

====================================================================== FAILURES ======================================================================
_______________________________________________ TestFactoryGeneratorWikibase.test_searchitem_language ________________________________________________

self = <tests.pagegenerators_tests.TestFactoryGeneratorWikibase testMethod=test_searchitem_language>

    def test_searchitem_language(self):
        """Test -searchitem with custom language specified."""
        gf = pagegenerators.GeneratorFactory(site=self.site)
        gf.handleArg('-searchitem:pl:abc')
        gf.handleArg('-limit:1')
        gen = gf.getCombinedGenerator()
        self.assertIsNotNone(gen)
        # ABC
        page1 = next(gen)
>       self.assertEqual(page1.title(), 'Q287078')
E       AssertionError: 'Q169889' != 'Q287078'
E       - Q169889
E       + Q287078

tests/pagegenerators_tests.py:1229: AssertionError
----------------------------------------------------------------- Captured log setup -----------------------------------------------------------------
connectionpool.py          824 DEBUG    Starting new HTTPS connection (1): www.wikidata.org
connectionpool.py          396 DEBUG    https://www.wikidata.org:443 "HEAD /w/index.php HTTP/1.1" 301 0
connectionpool.py          396 DEBUG    https://www.wikidata.org:443 "HEAD /wiki/Wikidata:Main_Page HTTP/1.1" 200 0
----------------------------------------------------------------- Captured log call ------------------------------------------------------------------
logging.py                 107 VERBOSE  Found 1 wikidata:wikidata processes running, including this one.
connectionpool.py          396 DEBUG    https://www.wikidata.org:443 "GET /w/api.php?action=query&meta=siteinfo%7Cuserinfo&siprop=namespaces%7Cnamespacealiases%7Cgeneral&continue=&uiprop=blockinfo%7Chasmsg&maxlag=5&format=json HTTP/1.1" 200 2119
connectionpool.py          396 DEBUG    https://www.wikidata.org:443 "GET /w/api.php?action=query&meta=siteinfo%7Cuserinfo&siprop=languages&continue=&uiprop=blockinfo%7Chasmsg&maxlag=5&format=json HTTP/1.1" 200 None
connectionpool.py          396 DEBUG    https://www.wikidata.org:443 "GET /w/api.php?action=paraminfo&modules=main%7Cparaminfo%7Cquery&maxlag=5&format=json HTTP/1.1" 200 None
connectionpool.py          396 DEBUG    https://www.wikidata.org:443 "GET /w/api.php?action=paraminfo&modules=wbsearchentities&maxlag=5&format=json HTTP/1.1" 200 1435
connectionpool.py          396 DEBUG    https://www.wikidata.org:443 "GET /w/api.php?search=abc&language=pl&action=wbsearchentities&continue=0&maxlag=5&format=json HTTP/1.1" 200 665

Change 401901 had a related patch set uploaded (by Rafidaslam; owner: rafid):
[pywikibot/core@master] pagegenerators_tests: Update test_searchitem_language

https://gerrit.wikimedia.org/r/401901

gerritbot added a project: Patch-For-Review.Jan 4 2018, 7:34 AM

rafidaslam claimed this task.Jan 4 2018, 7:35 AM

Change 401901 merged by jenkins-bot:
[pywikibot/core@master] pagegenerators_tests: Update test_searchitem_language

https://gerrit.wikimedia.org/r/401901

Dalba closed this task as Resolved.Jan 5 2018, 4:26 AM

Dalba triaged this task as High priority.