Page MenuHomePhabricator

Testing Pywikibot-Wikidata changes on non-production wikis
Open, HighPublic

Description

Pywikibot has some tests which use test.wikidata in https://github.com/wikimedia/pywikibot-core/tree/master/tests , however these test wikis appear to have a very odd configuration wrt integration tests. Where should people (such as Google-Code-in-2014 participants) do analysis / testing of integration of client and repo?

"You can use [[testwiki:|testwiki]] or [[test2wiki:|test2wiki]] an test clients." was added to the main page in March by user [[user:GZWDer]], and I guess that is true in a sense. However it doesnt appear possible to add sitelinks to testwiki or test2wiki, as far as I am aware.

Could someone confirm that (and that I am not going crazy)?

Is there an open Phab task about that? (I searched in Phab for "testwiki wikidata" but came up empty).

e.g.

  1. go to https://test2.wikipedia.org/wiki/0.8319452734493841_Moved
  2. click add sitelink / add a lang / link with page
  3. input language: extwiki
  4. input page: Wikipedia
  5. click 'Link with Page' to perform the operation

Result: error: "The external client site 'enwiki' did not provide page information for page '0.8319452734493841 Moved'."

Which is odd, I think..?

If the test wikis arnt able to be used together for integration testing, what other options are there?

  1. use test.wikidata as client and repo , or
  2. use beta.wmflabs.org, or
  3. ??

Event Timeline

jayvdb created this task.Dec 27 2014, 11:03 AM
jayvdb raised the priority of this task from to Needs Triage.
jayvdb updated the task description. (Show Details)
jayvdb added subscribers: jayvdb, Aklapper, Qgil.

Here is an example of the problem (after the rough patch on T85356 is applied)

$ python pwb.py scripts/harvest_template.py -family:wikipedia -lang:test -template:test test P404 -simulate
WARNING: cacheSources: Could not load sources: Page [[wikidata:test:Wikidata:List of wikis/python]] doesn't exist.
Finding redirects...
Retrieving 9 pages from wikipedia:test.
WARNING: API warning (wbgetentities): Unrecognized value for parameter 'sites': testwiki
ERROR: param-missing: A parameter that is required was missing (Either provide the item "ids" or pairs of "sites" and "titles" for corresponding pages)
Traceback (most recent call last):
  File "./pywikibot/bot.py", line 1190, in run
    item = pywikibot.ItemPage.fromPage(page)
  File "./pywikibot/page.py", line 3251, in fromPage
    if not lazy_load and not i.exists():
  File "./pywikibot/page.py", line 2867, in exists
    self.get()
  File "./pywikibot/page.py", line 3263, in get
    super(ItemPage, self).get(force=force, *args, **kwargs)
  File "./pywikibot/page.py", line 2901, in get
    data = self.repo.loadcontent(identification, *args)
  File "./pywikibot/site.py", line 5173, in loadcontent
    data = req.submit()
  File "./pywikibot/data/api.py", line 1093, in submit
    raise APIError(code, info, **result["error"])
APIError: param-missing: A parameter that is required was missing (Either provide the item "ids" or pairs of "sites" and "titles" for corresponding pages)
Qgil set Security to None.
Qgil added a subscriber: Quality-Assurance.
zeljkofilipin triaged this task as Medium priority.Mar 25 2015, 1:49 PM
jayvdb moved this task from Backlog to Framework on the Pywikibot-tests board.Jun 9 2015, 6:27 AM
jayvdb raised the priority of this task from Medium to High.Jun 17 2015, 6:35 AM
jayvdb added a project: Wikidata.

Is this still a problem? If not, can it be resolved?

Is this still a problem? If not, can it be resolved?

The steps to reproduce the problem still produce the same error.

Removing browser-tests, I do not think it is related. Please add it back if you think it is.

JanZerebecki moved this task from incoming to monitoring on the Wikidata board.Jul 23 2015, 5:05 PM
jayvdb added a comment.EditedAug 15 2015, 11:06 AM

So now this is a regular problem on pywikibot integration tests.
https://travis-ci.org/wikimedia/pywikibot-core/jobs/75715391#L1153

The test that is failing is test_unconnected in https://github.com/wikimedia/pywikibot-core/blob/master/tests/site_tests.py

  1. It loads https://test.wikipedia.org/wiki/Special:UnconnectedPages
  2. In the travis job error above it starts working on the first page on Special:UnconnectedPages, which is "Most-perfect magic square/invited"
  3. It loads the data repository for test.wikipedia, which is configured to be test.wikidata (see https://test.wikipedia.org/w/api.php?action=query&meta=wikibase)
  4. Tries to detect whether the page is indeed not linked to an item in the data repository using https://test.wikidata.org/w/api.php?format=json&action=wbgetentities&sites=testwiki&titles=Most-perfect%20magic%20square/invited
  5. The API responds with
{"servedby":"mw1017","warnings":{"wbgetentities":{"*":"Unrecognized value for parameter 'sites': testwiki"}},"error":{"code":"param-missing","info":"A parameter that is required was missing (Either provide the item \"ids\" or pairs of \"sites\" and \"titles\" for corresponding pages)","messages":[{"name":"wikibase-api-param-missing","parameters":[],"html":{"*":"A parameter that is required was missing"}}],"*":"See https://test.wikidata.org/w/api.php for API usage"}}

testwiki isnt a wiki listed as a valid value for sites: https://test.wikidata.org/w/api.php?action=help&modules=wbgetentities

The only test wiki which is listed as a value value for sites is : testwikidatawiki

Change 231764 had a related patch set uploaded (by John Vandenberg):
Wikibase integration tests fail on test.wikipedia

https://gerrit.wikimedia.org/r/231764

Change 231764 merged by jenkins-bot:
Wikibase integration tests fail on test.wikipedia

https://gerrit.wikimedia.org/r/231764

Adding a link at least works on test.wikpedia.org but this won't add a link for the site the link was made from.
Example:
On https://test.wikipedia.org/wiki/Blablabla i added https://de.wikipedia.org/wiki/Hauskatze but this created the item https://test.wikidata.org/wiki/Q1734 which links to https://en.wikipedia.org/wiki/Blah instead of testwiki.

Yes, it seems testwiki and test2wiki have an interesting configuration.

Preferred way of testing would be unit tests that don't talk to a mediawiki installation.
With what I wrote in T58961#1718288 you would get a job with a wiki, but as Wikibase doesn't work out of the box it needs additional commands to configure. In addition you'd need another wiki as a client. So that way would be a bit more work.
For local testing against a mediawiki the wikidata vagrant role should work out of the box (you get a wikifarm with an additional wikibase client wiki and repo wiki).

Qgil removed a subscriber: Qgil.Apr 24 2017, 1:53 PM