pywikibot should not retry if that doesn't make sense
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	WolfgangFahl
	Dec 18 2021, 4:55 PM

Description

List of steps to reproduce (step by step, including full links if applicable):
Try accessing a site with pywikibot that is misconfigured

What happens?:

requests.exceptions.ConnectionError: HTTPConnectionPool(host='api.php', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8b675f1610>: Failed to establish a new connection: [Errno -2] Name or service not known'))

What should have happened instead?:

no retries should be attempted if the Error is "Name or service not known" since that is probably not going away and the wait will take unnecessary long.

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc:
pywikibot 6.6.3

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	[IMPR] raise urllib3 NewConnectionError immediately	pywikibot/core	master	+9 -3

Customize query in gerrit

Event Timeline

WolfgangFahl created this task.Dec 18 2021, 4:55 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 18 2021, 4:55 PM

in the case a timeout occurs the original error messages should IMHO be made available as part of the exception and a hint given that this information is available. Otherwise it's IMHO to hard to debug such situations.

Zabe added a project: Pywikibot.Dec 18 2021, 5:33 PM

Restricted Application added a subscriber: pywikibot-bugs-list. · View Herald TranscriptDec 18 2021, 5:33 PM

Try accessing a site with pywikibot that is misconfigured

Do you have a sample for investigation?

Xqt claimed this task.Dec 19 2021, 5:31 PM

Xqt triaged this task as Medium priority.

Change 748375 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] raise urllib3 NewConnectionError immediately

https://gerrit.wikimedia.org/r/748375

gerritbot added a project: Patch-For-Review.Dec 19 2021, 5:36 PM

**Do you have a sample for investigation?
It turned out that i had set the scriptpath to "/" where an empty scriptpath was expected. I think even that this used to work in the past. The workaround was to set the scriptpath to "" again.

I think even that this used to work in the past.

A sample or url would be helpfull; I didn't found such a behaviour yet.

This is all i have. See https://github.com/WolfgangFahl/py-3rdparty-mediawiki/blob/067d2291e79c2f73726c177ac0dc3a889d43ef19/tests/test_SMWApi.py#L148. With scriptpath returning "/" the behavior shown in the log below happened. https://github.com/WolfgangFahl/py-3rdparty-mediawiki/commit/067d2291e79c2f73726c177ac0dc3a889d43ef19 fixed this - the family python file is generated and now has the changes.

cat wf_smwcopy.py 
# -*- coding: utf-8  -*-
from pywikibot import family

class Family(family.Family):
    name = 'smwcopy'
    langs = {
        'en': 'smw.bitplan.com',
    }
    def scriptpath(self, code):
       return ''
       
    def isPublic(self):
        return True   
        
    def version(self, code):
        return "1.35.0"  # The MediaWiki version used. Very important in most cases. (contrary to documentation)   

    def protocol(self, code):
       return 'http'

======================================================================
ERROR: testSMWAsk (tests.test_SMWApi.TestSMW)
test getting the unserialized json result of an ask query
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/tests/test_SMWApi.py", line 161, in testSMWAsk
    self.checkExpected(TestSMW.testask1,expectedRecords)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/tests/test_SMWApi.py", line 137, in checkExpected
    result=self.getAskResult(smw,ask)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/tests/test_SMWApi.py", line 129, in getAskResult
    result=smw.query(ask,limit=limit)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/wikibot/smw.py", line 603, in query
    rawresult=self.rawquery(ask,limit=limit)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/wikibot/smw.py", line 596, in rawquery
    result=self.submit(parameters)
  File "/home/runner/work/py-3rdparty-mediawiki/py-3rdparty-mediawiki/wikibot/smw.py", line 581, in submit
    return request.submit()
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/pywikibot/data/api.py", line 1758, in submit
    paramstring)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/pywikibot/data/api.py", line 1510, in _http_request
    self.wait()
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/pywikibot/data/api.py", line 1883, in wait
Error:     raise TimeoutError('Maximum retries attempted without success.')
pywikibot.exceptions.TimeoutError: Maximum retries attempted without success.

• BilalShirwani closed this task as a duplicate of T298058: [Impact Analysis] Evaluate Impact of Mobile Reply and New Discussion Tools.Dec 21 2021, 12:01 AM

JJMC89 reopened this task as Open.Dec 21 2021, 12:06 AM

@WolfgangFahl: I used generate_family_file which created

class Family(family.Family):  # noqa: D101

    name = 'smw'
    langs = {
        'en': 'smw.bitplan.com',
    }

    def scriptpath(self, code):
        return ''

    def protocol(self, code):
        return 'https'

The new behaviour after patch https://gerrit.wikimedia.org/r/c/pywikibot/core/+/748375 for the following test is like the matrix given below (after api cache was cleared):

import pywikibot
s = pywikibot.Site('en', fam='smw')
x = s.siteinfo['general']

protocol: ->	https	http
scriptpath:
''	passes	WARNING: Non-JSON response received, retries
'/'	raises ConnectionError immediately	raises ConnectionError immediately

It could pass if the cache wasn't cleared and the family file was changed after a valid request.

Is this new behviour what you expect?

Xqt closed this task as Resolved.Jan 1 2022, 1:07 PM

Change 748375 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] raise urllib3 NewConnectionError immediately

https://gerrit.wikimedia.org/r/748375

Maintenance_bot removed a project: Patch-For-Review.Jan 4 2022, 12:11 PM

Looks good to me in paper form.

pywikibot should not retry if that doesn't make senseClosed, ResolvedPublicBUG REPORTActions

Description

Details

Event Timeline

pywikibot should not retry if that doesn't make sense
Closed, ResolvedPublicBUG REPORT
Actions