Page MenuHomePhabricator

memento doctests fail since few days
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue:

from pywikibot.data.memento import MementoClient
mc = MementoClient()
dt = mc.convert_to_datetime("Sun, 01 Apr 2010 12:00:00 GMT")
datetime.datetime(2010, 4, 1, 12, 0)
mc = mc.get_memento_info("http://www.bbc.com/", dt)

https://integration.wikimedia.org/ci/job/pywikibot-core-tox-doctest-docker/5907/console

What happens?:

retrieving memento info fails and blocks CI on gerrit:

Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "C:\Python310\lib\http\client.py", line 1374, in getresponse
    response.begin()
  File "C:\Python310\lib\http\client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "C:\Python310\lib\http\client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "C:\Python310\lib\socket.py", line 705, in readinto
    return self._sock.recv_into(b)
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python310\lib\site-packages\requests\adapters.py", line 489, in send
    resp = conn.urlopen(
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "C:\Python310\lib\site-packages\urllib3\util\retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Python310\lib\site-packages\urllib3\packages\six.py", line 770, in reraise
    raise value
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 451, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 340, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='timetravel.mementoweb.org', port=80): Read timed out. (read timeout=9)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    mc = mc.get_memento_info("http://www.bbc.com/", dt)
  File "D:\pwb\GIT\core\pywikibot\data\memento.py", line 135, in get_memento_info
    tg_response = MementoClient.request_head(
  File "D:\pwb\GIT\core\pywikibot\data\memento.py", line 281, in request_head
    response = session.head(uri,
  File "C:\Python310\lib\site-packages\requests\sessions.py", line 622, in head
    return self.request("HEAD", url, **kwargs)
  File "C:\Python310\lib\site-packages\requests\sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python310\lib\site-packages\requests\sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python310\lib\site-packages\requests\adapters.py", line 578, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='timetravel.mementoweb.org', port=80): Read timed out. (read timeout=9)

The problem is inside memento.py. MementoClient.request_head call fails due to a TimeoutError:

tg_response = MementoClient.request_head(
    timegate_uri,
    accept_datetime=http_acc_dt,
    follow_redirects=True,
    session=self.session,
    timeout=timeout
)

Event Timeline

Xqt triaged this task as High priority.Jul 30 2022, 2:06 PM

Change 818535 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [tests] Temporary skip memento doc tests

https://gerrit.wikimedia.org/r/818535

Change 818535 merged by Xqt:

[pywikibot/core@master] [tests] Temporary skip memento doc tests

https://gerrit.wikimedia.org/r/818535

Xqt lowered the priority of this task from High to Low.Jul 30 2022, 2:14 PM
Xqt removed projects: Patch-For-Review, ci-test-error.
Xqt moved this task from Backlog to CI test failures on the Pywikibot-tests board.

Change 818537 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [tests] Increase timeout for memento doc tests

https://gerrit.wikimedia.org/r/818537

Change 818537 merged by jenkins-bot:

[pywikibot/core@master] [tests] Increase timeout for memento doc tests

https://gerrit.wikimedia.org/r/818537

Xqt claimed this task.