Page MenuHomePhabricator

Add botwiki as test site for MW 1.17
Closed, DeclinedPublic

Description

Botwiki [http://botwiki.sno.cc/wiki/Main_Page] has currently MW 1.17 and it is quite unused.
We could handshake with them not to upgrade the wiki and use it as test site with a lower MW version.

I was trying to run basic tests (I had to change botwiki_family.py removing self.namespaces dict) but I get:

>>> site = pywikibot.Site('botwiki', 'botwiki')
>>> page = pywikibot.Page(site, 'Main_Page')
>>> page
Page(Main Page)
>>> page.text
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/python/core/pywikibot/page.py", line 440, in text
    self._text = self.get(get_redirect=True)
  File "/home/user/python/core/pywikibot/tools.py", line 711, in wrapper
    return obj(*__args, **__kw)
  File "/home/user/python/core/pywikibot/page.py", line 349, in get
    self._getInternals(sysop)
  File "/home/user/python/core/pywikibot/page.py", line 373, in _getInternals
    self.site.loadrevisions(self, getText=True, sysop=sysop)
  File "/home/user/python/core/pywikibot/site.py", line 3159, in loadrevisions
    step=step, total=total, **rvargs)
  File "/home/user/python/core/pywikibot/site.py", line 1587, in _generator
    gen = gen_class(type_arg, site=self, **args)
  File "/home/user/python/core/pywikibot/data/api.py", line 1689, in __init__
    QueryGenerator.__init__(self, prop=prop, **kwargs)
  File "/home/user/python/core/pywikibot/data/api.py", line 1299, in __init__
    set(self.modules) & self.site._paraminfo.query_modules_with_limits
  File "/home/user/python/core/pywikibot/data/api.py", line 437, in query_modules_with_limits
    self.fetch(self.query_modules)
  File "/home/user/python/core/pywikibot/data/api.py", line 402, in query_modules
    self._init()
  File "/home/user/python/core/pywikibot/data/api.py", line 218, in _init
    assert('type' in query_modules_param)
AssertionError

At first site it looks paraminfo has some issue with that version (?).

Event Timeline

Mpaa raised the priority of this task from to Needs Triage.
Mpaa updated the task description. (Show Details)
Mpaa added a project: Pywikibot.
Mpaa updated the task description. (Show Details)
Mpaa set Security to None.
Mpaa subscribed.
Mpaa renamed this task from botwiki as test site for MW 1.19, Paraminfo throws Exception to botwiki as test site for MW 1.17, Paraminfo throws Exception.Jan 5 2015, 10:46 AM
Mpaa updated the task description. (Show Details)

Change 182088 had a related patch set uploaded (by John Vandenberg):
Add ParamInfo support for v1.15-18

https://gerrit.wikimedia.org/r/182088

Patch-For-Review

Change 182088 merged by jenkins-bot:
Add ParamInfo support for v1.15-18

https://gerrit.wikimedia.org/r/182088

The "Paraminfo throws Exception" is solved by https://gerrit.wikimedia.org/r/182088.

The rest of the task is still open.
If botwiki will turn out not OK as test site for some reason (not wanted, available or suitable, etc.), then this task can be closed.

The idea is that we add botwiki to the pwb .travis.yml build grid, which will mean it is hit regularly with a batch of tests

https://travis-ci.org/wikimedia/pywikibot-core/builds

But that only works if botwiki doesnt mind, and stays on 1.17 ;-)

We have a similar task (T75513) to use Wikia which is 1.19 .

valhallasw renamed this task from botwiki as test site for MW 1.17, Paraminfo throws Exception to Add botwiki as test site for MW 1.17.Jan 29 2015, 9:24 PM
valhallasw triaged this task as Low priority.

A test run against botwiki

https://travis-ci.org/jayvdb/pywikibot-core/jobs/67976315

two errors:

ERROR: test_deprecated_token (tests.site_tests.TestSiteTokens)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/travis/build/jayvdb/pywikibot-core/tests/site_tests.py", line 1618, in test_deprecated_token
    self.assertEqual(self.mysite.getPatrolToken(), self.mysite.tokens['patrol'])
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/tools/__init__.py", line 1136, in wrapper
    return obj(*args, **kwargs)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 2957, in getPatrolToken
    return self.tokens['patrol']
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 1515, in __getitem__
    .format(key, self.site.user(), self.site))
Error: Action 'patrol' is not allowed for user JVbot-test on botwiki:en wiki.
======================================================================
FAIL: Test the site.allimages() method.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/jayvdb/pywikibot-core/tests/site_tests.py", line 645, in testAllImages
    self.assertTrue(mysite.page_exists(impage))
AssertionError: False is not true

[[botwiki:File:12345.JPG]] was not existing, recreated now.

$ python -m unittest tests.site_tests.TestSiteGenerators.testAllImages
tests: max_retries reduced from 25 to 1
.
----------------------------------------------------------------------
Ran 1 test in 5.958s

OK

Change 220298 had a related patch set uploaded (by Mpaa):
site_tests.py: skip patrol test if wiki MWV is too low

https://gerrit.wikimedia.org/r/220298

Change 220298 merged by jenkins-bot:
site_tests.py: skip deprecated patrol test if user is not allowed

https://gerrit.wikimedia.org/r/220298

Hmm. I run it again with those fixes, and more errors now :/

https://travis-ci.org/jayvdb/pywikibot-core/jobs/68085027

======================================================================
ERROR: testLinks (tests.page_tests.TestPageObject)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/jayvdb/pywikibot-core/tests/page_tests.py", line 469, in testLinks
    for p in mainpage.langlinks():
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/page.py", line 1300, in langlinks
    self._langlinks = list(self.iterlanglinks(include_obsolete=True))
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 3467, in pagelanglinks
    source=self)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/page.py", line 5022, in langlinkUnsafe
    link._site = pywikibot.Site(lang, source.family.name)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/__init__.py", line 615, in Site
    _sites[key] = interface(code=code, fam=fam, user=user, sysop=sysop)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 1559, in __init__
    BaseSite.__init__(self, code, fam, user, sysop)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 563, in __init__
    % (self.__code, self.__family.name))
UnknownSite: Language 'it' does not exist in family botwiki

======================================================================
ERROR: test_purge (tests.page_tests.TestPageUserAction)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/jayvdb/pywikibot-core/tests/page_tests.py", line 744, in test_purge
    self.assertIsInstance(mainpage.purge(), bool)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/page.py", line 1206, in purge
    return self.site.purgepages([self], **kwargs)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 1101, in callee
    return fn(self, *args, **kwargs)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 4986, in purgepages
    result = req.submit()
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/data/api.py", line 1877, in submit
    raise APIError(**result['error'])
APIError: writeapidenied: You're not allowed to edit this wiki through the API

======================================================================
ERROR: Test Page.watch, with and without unwatch enabled.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/jayvdb/pywikibot-core/tests/page_tests.py", line 751, in test_watch
    rv = userpage.watch()
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/page.py", line 1199, in watch
    return self.site.watchpage(self, unwatch)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 1101, in callee
    return fn(self, *args, **kwargs)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 4960, in watchpage
    'token': self.tokens['watch'],
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 1501, in __getitem__
    .format(key, self.site))
Error: Requested token 'watch' is invalid on botwiki:en wiki.

======================================================================
ERROR: Test the site.patrol() method.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/jayvdb/pywikibot-core/tests/site_tests.py", line 1461, in testPatrol
    result = list(mysite.patrol(rcid=rc['rcid']))
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 4853, in patrol
    token = self.tokens['patrol']
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 1515, in __getitem__
    .format(key, self.site.user(), self.site))
Error: Action 'patrol' is not allowed for user JVbot-test on botwiki:en wiki.

======================================================================
ERROR: Test site methods for getting links to and from a page.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/jayvdb/pywikibot-core/tests/site_tests.py", line 448, in testLinkMethods
    for ll in mysite.pagelanglinks(mainpage):
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 3467, in pagelanglinks
    source=self)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/page.py", line 5022, in langlinkUnsafe
    link._site = pywikibot.Site(lang, source.family.name)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/__init__.py", line 615, in Site
    _sites[key] = interface(code=code, fam=fam, user=user, sysop=sysop)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 1559, in __init__
    BaseSite.__init__(self, code, fam, user, sysop)
  File "/home/travis/build/jayvdb/pywikibot-core/pywikibot/site.py", line 563, in __init__
    % (self.__code, self.__family.name))
UnknownSite: Language 'it' does not exist in family botwiki

The account 'Pywikibot-test' now exists on botwiki, so we can add it to the build matrix whenever the tests pass. However not all dev people will want to create accounts on botwiki to run the travis builds, so this is another situation similar to T100802 where the same username may not exist on all wikis on the matrix.

Change 214816 had a related patch set uploaded (by John Vandenberg):
Do not ask for password if user does not exist

https://gerrit.wikimedia.org/r/214816

[[botwiki:File:12345.JPG]] was not existing, recreated now.

$ python -m unittest tests.site_tests.TestSiteGenerators.testAllImages
tests: max_retries reduced from 25 to 1
.
----------------------------------------------------------------------
Ran 1 test in 5.958s

OK

But why was it iterating over that page when it doesn't exist? Since when does query+allimages return missing files?

@XZise, I do not know. The page was deleted and then restored, but that was not enough, so it still needed to be created.
See history:
19:29, 23 June 2015 Mpaa (Talk | contribs) uploaded a new version of "File:12345.JPG" ‎ (test)
13:35, 29 April 2011 Marco Plassio (Talk | contribs) restored "File:12345.JPG" ‎ (1 file(s) restored: errore mio) -> here page.exists() was still False.
13:33, 29 April 2011 Marco Plassio (Talk | contribs) deleted "File:12345.JPG" ‎
03:40, 18 June 2010 AssaSom (Talk | contribs) uploaded "File:12345.JPG" ‎

Hmm okay. Did you check what the API reported and if there is a problem with pywikibot or with the MW API?

I ran botwiki.sno.cc/w/api.php?action=query&titles=File:12345.JPG&prop=info
By heart I do not remember, but was something like this ...I remember in pywikibot page._revid=0.

<api>
  <query>
    <pages>
        <page ns="6" title="File:12345.JPG" missing="">  --> not sure about this line but definitely not all the info of an existing page were there
      </page>
    </pages>
  </query>
</api>

@XZise, I asked to restore another deleted image:
19:52, 24 June 2015 Nemo bis (Talk | contribs) restored "File:0IBBK.gif" ‎ (1 revision and 1 file restored: Temporary restore, asked by Mpaa for some test on File:12345.JPG)
19:54, 27 November 2011 Snowolf (Talk | contribs) deleted "File:0IBBK.gif" ‎
19:50, 27 November 2011 Snowolf (Talk | contribs) uploaded "File:0IBBK.gif" ‎ (Testing)

Here it says: 1 revision and 1 file restored vs. (1 file(s) restored: errore mio) in the previous case.
So probably something was strange on how the other image was restored.

Just for reference this is the output:

<?xml version="1.0"?>
<api>
  <query>
    <pages>
      <page pageid="2615" ns="6" title="File:0IBBK.gif" touched="2015-06-24T19:52:29Z" lastrevid="8126" counter="5" length="7" new="" />
    </pages>
  </query>
</api>

Change 214816 merged by jenkins-bot:
Do not ask for password if user does not exist

https://gerrit.wikimedia.org/r/214816

Xqt subscribed.

botwiki is not available for months.