Page MenuHomePhabricator

Tests fail with test.wikipedia.org family
Closed, InvalidPublic

Description

Commit 66d88aa removed our test.wikipedia.org family. I'm unsure why we did that, so I ran generate_family_file.py to safely run our scripts without breaking live pages. Unfortunately doing so breaks our tests.

Here's my family config...

% cat pywikibot/families/test_family.py
"""
This family file was auto-generated by generate_family_file.py script.

Configuration parameters:
  url = https://test.wikipedia.org/
  name = test

Please do not commit this to the Git repository!
"""
from pywikibot import family


class Family(family.Family):  # noqa: D101

    name = 'test'
    langs = {
        'en': 'test.wikipedia.org',
    }

    def scriptpath(self, code):
        return {
            'en': '/w',
        }[code]

    def protocol(self, code):
        return {
            'en': 'https',
        }[code]

And here's the test failures...

% time python -m unittest
...
======================================================================
ERROR: test_preload_langlinks_count (tests.site_tests.TestPagePreloading)
Test preloading continuation works.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.8/unittest/mock.py", line 1325, in patched
    return func(*newargs, **newkeywargs)
  File "/home/atagar/Desktop/pywikibot/tests/site_tests.py", line 3144, in test_preload_langlinks_count
    pages = list(gen)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_generators.py", line 195, in preloadpages
    api.update_page(page, pagedata, rvgen.props)
  File "/home/atagar/Desktop/pywikibot/pywikibot/data/api.py", line 3145, in update_page
    _update_langlinks(page, pagedict['langlinks'])
  File "/home/atagar/Desktop/pywikibot/pywikibot/data/api.py", line 3069, in _update_langlinks
    links = [pywikibot.Link.langlinkUnsafe(link['lang'], link['*'],
  File "/home/atagar/Desktop/pywikibot/pywikibot/data/api.py", line 3069, in <listcomp>
    links = [pywikibot.Link.langlinkUnsafe(link['lang'], link['*'],
  File "/home/atagar/Desktop/pywikibot/pywikibot/page/__init__.py", line 5610, in langlinkUnsafe
    link._site = pywikibot.Site(lang, source.family.name)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 404, in wrapper
    return obj(*__args, **__kw)
  File "/home/atagar/Desktop/pywikibot/pywikibot/__init__.py", line 1207, in Site
    _sites[key] = interface(code=code, fam=fam, user=user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_apisite.py", line 115, in __init__
    super().__init__(code, fam, user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_basesite.py", line 90, in __init__
    raise UnknownSiteError(error_msg)
pywikibot.exceptions.UnknownSiteError: Language 'af' does not exist in family test

======================================================================
ERROR: test_preload_langlinks_normal (tests.site_tests.TestPagePreloading)
Test preloading langlinks works.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/site_tests.py", line 3123, in test_preload_langlinks_normal
    for count, page in enumerate(gen):
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_generators.py", line 195, in preloadpages
    api.update_page(page, pagedata, rvgen.props)
  File "/home/atagar/Desktop/pywikibot/pywikibot/data/api.py", line 3145, in update_page
    _update_langlinks(page, pagedict['langlinks'])
  File "/home/atagar/Desktop/pywikibot/pywikibot/data/api.py", line 3069, in _update_langlinks
    links = [pywikibot.Link.langlinkUnsafe(link['lang'], link['*'],
  File "/home/atagar/Desktop/pywikibot/pywikibot/data/api.py", line 3069, in <listcomp>
    links = [pywikibot.Link.langlinkUnsafe(link['lang'], link['*'],
  File "/home/atagar/Desktop/pywikibot/pywikibot/page/__init__.py", line 5610, in langlinkUnsafe
    link._site = pywikibot.Site(lang, source.family.name)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 404, in wrapper
    return obj(*__args, **__kw)
  File "/home/atagar/Desktop/pywikibot/pywikibot/__init__.py", line 1207, in Site
    _sites[key] = interface(code=code, fam=fam, user=user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_apisite.py", line 115, in __init__
    super().__init__(code, fam, user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_basesite.py", line 90, in __init__
    raise UnknownSiteError(error_msg)
pywikibot.exceptions.UnknownSiteError: Language 'af' does not exist in family test

======================================================================
ERROR: test_allpages_langlinks_enabled (tests.site_tests.TestSiteGenerators)
Test allpages with langlinks enabled.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/site_tests.py", line 517, in test_allpages_langlinks_enabled
    self.assertNotEqual(page.langlinks(), [])
  File "/home/atagar/Desktop/pywikibot/pywikibot/page/__init__.py", line 1448, in langlinks
    self._langlinks = list(self.iterlanglinks(include_obsolete=True))
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_generators.py", line 759, in pagelanglinks
    link = pywikibot.Link.langlinkUnsafe(linkdata['lang'],
  File "/home/atagar/Desktop/pywikibot/pywikibot/page/__init__.py", line 5610, in langlinkUnsafe
    link._site = pywikibot.Site(lang, source.family.name)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 404, in wrapper
    return obj(*__args, **__kw)
  File "/home/atagar/Desktop/pywikibot/pywikibot/__init__.py", line 1207, in Site
    _sites[key] = interface(code=code, fam=fam, user=user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_apisite.py", line 115, in __init__
    super().__init__(code, fam, user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_basesite.py", line 90, in __init__
    raise UnknownSiteError(error_msg)
pywikibot.exceptions.UnknownSiteError: Language 'af' does not exist in family test

======================================================================
ERROR: test_pagelanglinks (tests.site_tests.TestSiteGenerators)
Test Site.pagelanglinks.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/site_tests.py", line 427, in test_pagelanglinks
    for ll in self.site.pagelanglinks(self.mainpage):
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_generators.py", line 759, in pagelanglinks
    link = pywikibot.Link.langlinkUnsafe(linkdata['lang'],
  File "/home/atagar/Desktop/pywikibot/pywikibot/page/__init__.py", line 5610, in langlinkUnsafe
    link._site = pywikibot.Site(lang, source.family.name)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 404, in wrapper
    return obj(*__args, **__kw)
  File "/home/atagar/Desktop/pywikibot/pywikibot/__init__.py", line 1207, in Site
    _sites[key] = interface(code=code, fam=fam, user=user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_apisite.py", line 115, in __init__
    super().__init__(code, fam, user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_basesite.py", line 90, in __init__
    raise UnknownSiteError(error_msg)
pywikibot.exceptions.UnknownSiteError: Language 'af' does not exist in family test

======================================================================
ERROR: testLinks (tests.page_tests.TestPageObject)
Test the different types of links from a page.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/page_tests.py", line 541, in testLinks
    for p in mainpage.langlinks():
  File "/home/atagar/Desktop/pywikibot/pywikibot/page/__init__.py", line 1448, in langlinks
    self._langlinks = list(self.iterlanglinks(include_obsolete=True))
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_generators.py", line 759, in pagelanglinks
    link = pywikibot.Link.langlinkUnsafe(linkdata['lang'],
  File "/home/atagar/Desktop/pywikibot/pywikibot/page/__init__.py", line 5610, in langlinkUnsafe
    link._site = pywikibot.Site(lang, source.family.name)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 404, in wrapper
    return obj(*__args, **__kw)
  File "/home/atagar/Desktop/pywikibot/pywikibot/__init__.py", line 1207, in Site
    _sites[key] = interface(code=code, fam=fam, user=user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_apisite.py", line 115, in __init__
    super().__init__(code, fam, user)
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 487, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_basesite.py", line 90, in __init__
    raise UnknownSiteError(error_msg)
pywikibot.exceptions.UnknownSiteError: Language 'af' does not exist in family test

======================================================================
FAIL: test_search_where_title (tests.site_tests.SearchTestCase)
Test site.search() method with 'where' parameter set to title.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/__init__.py", line 196, in suppressed_func
    return func(*args, **kwargs)
  File "/home/atagar/Desktop/pywikibot/tests/site_tests.py", line 1459, in test_search_where_title
    self.assertEqual(search_gen.request._params, expected_params)
AssertionError: {'gsrsearch': ['wiki'], 'gsrwhat': ['title'[280 chars] [0]} != {'prop': ['info', 'imageinfo', 'categoryinf[285 chars]one]}
  {'action': ['query'],
   'continue': [True],
   'generator': ['search'],
   'gsrnamespace': [0],
-  'gsrsearch': ['wiki'],
+  'gsrsearch': ['intitle:wiki'],
?                 ++++++++

-  'gsrwhat': ['title'],
?              ^^^^^ -

+  'gsrwhat': [None],
?              ^^^

   'iilimit': ['max'],
   'iiprop': ['timestamp', 'user', 'comment', 'url', 'size', 'sha1', 'metadata'],
   'indexpageids': [True],
   'inprop': ['protection'],
   'prop': ['info', 'imageinfo', 'categoryinfo']}

======================================================================
FAIL: test_basic (tests.site_tests.TestAlldeletedrevisionsAsUser)
Test the site.alldeletedrevisions() method.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/site_tests.py", line 1616, in test_basic
    self.assertTrue(all('revisions' in data
AssertionError: False is not true

======================================================================
FAIL: test_constructors (tests.site_tests.TestSiteObject) (dbname='testwiki')
Test cases for site constructors.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/site_tests.py", line 128, in test_constructors
    self.assertIs(
AssertionError: APISite("en", "test") is not APISite("test", "wikipedia")

======================================================================
FAIL: test_each_family (tests.family_tests.TestFamilyUrlRegex) (family='test')
Test each family builds a working regex.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/family_tests.py", line 235, in test_each_family
    self.assertEqual(family.from_url(url), code)
  File "/home/atagar/Desktop/pywikibot/pywikibot/family.py", line 864, in from_url
    site = pywikibot.Site(code, self.name)
  File "/home/atagar/Desktop/pywikibot/tests/family_tests.py", line 161, in Site
    self.assertEqual(fam, 'wikipedia')
AssertionError: 'test' != 'wikipedia'
- test
+ wikipedia


======================================================================
FAIL: test_fully_qualified_NS0_family (tests.link_tests.TestFullyQualifiedNoLangFamilyExplicitLinkParser)
Test ':wikidata:testwiki:Q6' on enwp is namespace 0.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/link_tests.py", line 540, in test_fully_qualified_NS0_family
    self.assertEqual(link.site, self.get_site('test.wp'))
AssertionError: APISite("en", "test") != APISite("test", "wikipedia")

======================================================================
FAIL: test_fully_qualified_NS1_family (tests.link_tests.TestFullyQualifiedNoLangFamilyExplicitLinkParser)
Test ':wikidata:testwiki:Talk:Q6' on enwp is namespace 1.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/link_tests.py", line 548, in test_fully_qualified_NS1_family
    self.assertEqual(link.site, self.get_site('test.wp'))
AssertionError: APISite("en", "test") != APISite("test", "wikipedia")

======================================================================
FAIL: test_fully_qualified_NS0 (tests.link_tests.TestFullyQualifiedNoLangFamilyImplicitLinkParser) (pattern='wikidata:testwiki')
Test prefixed links with 'Q6' on enwp is namespace 0.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/link_tests.py", line 804, in test_fully_qualified_NS0
    self.assertEqual(link.site, pywikibot.Site(sitetitle))
AssertionError: APISite("en", "test") != APISite("test", "wikipedia")

======================================================================
FAIL: test_fully_qualified_NS1 (tests.link_tests.TestFullyQualifiedNoLangFamilyImplicitLinkParser) (pattern='wikidata:testwiki')
Test prefixed links with 'Talk:Q6' on enwp is namespace 1.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/link_tests.py", line 816, in test_fully_qualified_NS1
    self.assertEqual(link.site, pywikibot.Site(sitetitle))
AssertionError: APISite("en", "test") != APISite("test", "wikipedia")

----------------------------------------------------------------------
Ran 1771 tests in 630.414s

FAILED (failures=8, errors=5, skipped=107, expected failures=10, unexpected successes=1)

real	10m31.700s
user	2m20.211s
sys	0m12.341s

Tests should not rely on the user's local configuration. I have fixes for a few of these but quite yet for all of 'em.

Event Timeline

Change 707865 had a related patch set uploaded (by Damian; author: Damian):

[pywikibot/core@master] [bugfix] Tests fail if family is named 'test'

https://gerrit.wikimedia.org/r/707865

Change 707867 had a related patch set uploaded (by Damian; author: Damian):

[pywikibot/core@master] [bugfix] Drop test_constructors test wiki checks

https://gerrit.wikimedia.org/r/707867

Change 707868 had a related patch set uploaded (by Damian; author: Damian):

[pywikibot/core@master] [bugfix] Fix CirrusSearch test conditional

https://gerrit.wikimedia.org/r/707868

Change 707869 had a related patch set uploaded (by Damian; author: Damian):

[pywikibot/core@master] [bugfix] Broaden TestAlldeletedrevisionsAsUser assertions

https://gerrit.wikimedia.org/r/707869

test.wikipedia.org is part of wikipedia_family.py. Therefore the test family was dropped. See T228375

Change 707865 merged by jenkins-bot:

[pywikibot/core@master] [bugfix] Tests fail if family is named 'test'

https://gerrit.wikimedia.org/r/707865

Xqt requested that I move information about https://gerrit.wikimedia.org/r/c/pywikibot/core/+/707869 onto this ticket. The 'drev' value with the family mentioned above is...

[
  {   
    'pageid': 120329,
    'revisions': [
      {
        'revid': 475999,
        'parentid': 0,
        'user': 'Atagar-bot',
        'timestamp': '2021-06-29T00:29:33Z'
      }
    ],  
    'ns': 0,
    'title': 'Pywikibot move test 4'
  }   
]

Which causes...

% python -m tests.site_tests TestAlldeletedrevisionsAsUser
F.....
======================================================================
FAIL: test_basic (__main__.TestAlldeletedrevisionsAsUser)
Test the site.alldeletedrevisions() method.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "pywikibot/tests/site_tests.py", line 1616, in test_basic
    self.assertTrue(all('revisions' in data
AssertionError: False is not true

test.wikipedia.org is part of wikipedia_family.py. Therefore the test family was dropped. See T228375

Hi Xqt. I read T228375 but it doesn't explain how to configure myself to use it. Is this explained somewhere on our readme or site? If not we should probably add it.

Change 707867 abandoned by Damian:

[pywikibot/core@master] [bugfix] Drop test_constructors test wiki checks

Reason:

Local family file triggered incorrect disambiguation.

https://gerrit.wikimedia.org/r/707867

test.wikipedia.org is part of wikipedia_family.py. Therefore the test family was dropped. See T228375

Hi Xqt. I read T228375 but it doesn't explain how to configure myself to use it. Is this explained somewhere on our readme or site? If not we should probably add it.

Just use Site('wikipedia:test') for test.wikipedia.org. The old test_family just duplicated the same site. (Ok the test family was prior)

Xqt requested that I move information about https://gerrit.wikimedia.org/r/c/pywikibot/core/+/707869 onto this ticket. The 'drev' value with the family mentioned above is...

What is meant with drev value? How to get it? Where is it described in mediawiki API? Was there any change in structure?

Xqt closed this task as Invalid.EditedJul 26 2021, 3:07 PM

Creating a family file for test.wikipedia.org collides with wikipedia_family. You cannot have a solution if its base is wrong. The test works as expected:

C:\pwb\GIT\core>py -3.9 pwb.py site_tests -v TestAlldeletedrevisionsAsUser.test_basic
tests: max_retries reduced from 15 to 1
test_basic (__main__.TestAlldeletedrevisionsAsUser)
Test the site.alldeletedrevisions() method. ... ok

----------------------------------------------------------------------
Ran 1 test in 2.085s

OK

C:\pwb\GIT\core>

Just use Site('wikipedia:test') for test.wikipedia.org. The old test_family just duplicated the same site. (Ok the test family was prior)

Thanks Xqt. My question was how to run our scripts with test.wikipedia.org but I figured it out. The trick is to change my user-config.py's 'mylang' to test...

% cat user-config.py
family = 'wikipedia'
mylang = 'test'
usernames['wikipedia']['test'] = 'Atagar-bot'
... etc...

Personally I think this is pretty unintuitive and "How can practice running scripts with test.wikipedia.org?" would be a good addition if we have a FAQ somewhere.

Most of my test failures are now resolves so closing.

Change 707868 abandoned by Damian:

[pywikibot/core@master] [bugfix] Fix CirrusSearch test conditional

Reason:

Resolved by removing extra family file (see T287332)

https://gerrit.wikimedia.org/r/707868

Hi Xqt. Sorry to reopen but changing my user-config.py in the way mentioned above surfaced a new test failure and I think this one is legitimate...

% python -m tests.page_tests TestShortLink
======================================================================
ERROR: test_create_short_link (__main__.TestShortLink)
Test create_short_link function.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/pywikibot/login.py", line 110, in __init__
    user = code_to_usr.get(site.code) or code_to_usr['*']
KeyError: '*'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/atagar/Desktop/pywikibot/tests/page_tests.py", line 1267, in test_create_short_link
    meta.login()
  File "/home/atagar/Desktop/pywikibot/pywikibot/site/_apisite.py", line 389, in login
    login_manager = api.LoginManager(site=self, user=self.username())
  File "/home/atagar/Desktop/pywikibot/pywikibot/tools/_deprecate.py", line 404, in wrapper
    return obj(*__args, **__kw)
  File "/home/atagar/Desktop/pywikibot/pywikibot/login.py", line 112, in __init__
    raise NoUsernameError(
pywikibot.exceptions.NoUsernameError: ERROR: username for meta:meta is undefined.
If you have a username for that site, please add a line to user-config.py as follows:
usernames['meta']['meta'] = 'myUsername'

The problem looks to be that the test is configured to only run the user has 'wikipedia:test' login credentials, but attempts to log into 'meta:meta' instead. I'll shoot you a patch in a sec.

Change 708335 had a related patch set uploaded (by Damian; author: Damian):

[pywikibot/core@master] [bugfix] TestShortLink requires meta:meta credentials

https://gerrit.wikimedia.org/r/708335

Change 708335 abandoned by Damian:

[pywikibot/core@master] [bugfix] TestShortLink requires meta:meta credentials

Reason:

Xqt says this test behavior is intentional.

https://gerrit.wikimedia.org/r/708335