Page MenuHomePhabricator

pywikibot should not retry if that doesn't make sense
Closed, ResolvedPublicBUG REPORT

Description

List of steps to reproduce (step by step, including full links if applicable):
Try accessing a site with pywikibot that is misconfigured

What happens?:

requests.exceptions.ConnectionError: HTTPConnectionPool(host='api.php', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8b675f1610>: Failed to establish a new connection: [Errno -2] Name or service not known'))

What should have happened instead?:

no retries should be attempted if the Error is "Name or service not known" since that is probably not going away and the wait will take unnecessary long.

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc:
pywikibot 6.6.3

Event Timeline

in the case a timeout occurs the original error messages should IMHO be made available as part of the exception and a hint given that this information is available. Otherwise it's IMHO to hard to debug such situations.

Try accessing a site with pywikibot that is misconfigured

Do you have a sample for investigation?

Xqt triaged this task as Medium priority.

Change 748375 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] raise urllib3 NewConnectionError immediately

https://gerrit.wikimedia.org/r/748375

**Do you have a sample for investigation?
It turned out that i had set the scriptpath to "/" where an empty scriptpath was expected. I think even that this used to work in the past. The workaround was to set the scriptpath to "" again.

I think even that this used to work in the past.

A sample or url would be helpfull; I didn't found such a behaviour yet.

This is all i have. See https://github.com/WolfgangFahl/py-3rdparty-mediawiki/blob/067d2291e79c2f73726c177ac0dc3a889d43ef19/tests/test_SMWApi.py#L148. With scriptpath returning "/" the behavior shown in the log below happened. https://github.com/WolfgangFahl/py-3rdparty-mediawiki/commit/067d2291e79c2f73726c177ac0dc3a889d43ef19 fixed this - the family python file is generated and now has the changes.

cat wf_smwcopy.py 
# -*- coding: utf-8  -*-
from pywikibot import family

class Family(family.Family):
    name = 'smwcopy'
    langs = {
        'en': 'smw.bitplan.com',
    }
    def scriptpath(self, code):
       return ''
       
    def isPublic(self):
        return True   
        
    def version(self, code):
        return "1.35.0"  # The MediaWiki version used. Very important in most cases. (contrary to documentation)   

    def protocol(self, code):
       return 'http'
======================================================================
ERROR: testSMWAsk (tests.test_SMWApi.TestSMW)
test getting the unserialized json result of an ask query
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/tests/test_SMWApi.py", line 161, in testSMWAsk
    self.checkExpected(TestSMW.testask1,expectedRecords)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/tests/test_SMWApi.py", line 137, in checkExpected
    result=self.getAskResult(smw,ask)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/tests/test_SMWApi.py", line 129, in getAskResult
    result=smw.query(ask,limit=limit)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/wikibot/smw.py", line 603, in query
    rawresult=self.rawquery(ask,limit=limit)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/wikibot/smw.py", line 596, in rawquery
    result=self.submit(parameters)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/wikibot/smw.py", line 581, in submit
    return request.submit()
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/pywikibot/data/api.py", line 1758, in submit
    paramstring)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/pywikibot/data/api.py", line 1510, in _http_request
    self.wait()
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/pywikibot/data/api.py", line 1883, in wait
Error:     raise TimeoutError('Maximum retries attempted without success.')
pywikibot.exceptions.TimeoutError: Maximum retries attempted without success.

@WolfgangFahl: I used generate_family_file which created

class Family(family.Family):  # noqa: D101

    name = 'smw'
    langs = {
        'en': 'smw.bitplan.com',
    }

    def scriptpath(self, code):
        return ''

    def protocol(self, code):
        return 'https'

The new behaviour after patch https://gerrit.wikimedia.org/r/c/pywikibot/core/+/748375 for the following test is like the matrix given below (after api cache was cleared):

import pywikibot
s = pywikibot.Site('en', fam='smw')
x = s.siteinfo['general']
protocol: ->httpshttp
scriptpath:
''passesWARNING: Non-JSON response received, retries
'/'raises ConnectionError immediatelyraises ConnectionError immediately

It could pass if the cache wasn't cleared and the family file was changed after a valid request.

Is this new behviour what you expect?

Change 748375 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] raise urllib3 NewConnectionError immediately

https://gerrit.wikimedia.org/r/748375