Page MenuHomePhabricator

Travis builds halting and failing on test_recentchanges_default / test_recentchanges_ns_default
Closed, ResolvedPublic

Description

The travis test jobs that do not set PYWIKIBOT2_TEST_NO_RC=1 (i.e. most of the odd number jobs) are failing with

Test recentchanges generator with default setting. ... 

No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.

The build has been terminated

There are also many builds with a different error on the same test, which might be occurring before the 10 minute timeout:

======================================================================
ERROR: test_recentchanges_ns_default (tests.pagegenerators_tests.TestFactoryGenerator)
Test recentchanges generator.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/wikimedia/pywikibot-core/tests/pagegenerators_tests.py", line 821, in test_recentchanges_ns_default
    self.assertPagesInNamespacesAll(gen, set([0, 1, 2]), skip=True)
  File "/home/travis/build/wikimedia/pywikibot-core/tests/aspects.py", line 223, in assertPagesInNamespacesAll
    page_namespaces = [page.namespace() for page in gen]
  File "/home/travis/build/wikimedia/pywikibot-core/pywikibot/tools/__init__.py", line 801, in filter_unique
    if (key(item) if key else item) not in container:
  File "/home/travis/build/wikimedia/pywikibot-core/pywikibot/page.py", line 347, in __hash__
    return hash(unicode(self))
  File "/home/travis/build/wikimedia/pywikibot-core/pywikibot/page.py", line 315, in __unicode__
    return self.title(asLink=True, forceInterwiki=True)
  File "/home/travis/build/wikimedia/pywikibot-core/pywikibot/tools/__init__.py", line 1370, in wrapper
    return obj(*__args, **__kw)
  File "/home/travis/build/wikimedia/pywikibot-core/pywikibot/page.py", line 250, in title
    title = self._link.canonical_title()
  File "/home/travis/build/wikimedia/pywikibot-core/pywikibot/page.py", line 5154, in canonical_title
    if self.namespace != Namespace.MAIN:
  File "/home/travis/build/wikimedia/pywikibot-core/pywikibot/page.py", line 5115, in namespace
    self.parse()
  File "/home/travis/build/wikimedia/pywikibot-core/pywikibot/page.py", line 5041, in parse
    u"'{0}' has no title.".format(self._text))
InvalidTitle: 'User:#6573724' has no title.

There are a lot of developer builds on travis at the moment, making it a bit more difficult to find where the breakage started. Here are the recent master builds, with confirmed problems ticked

  • # 3407 .1 (test_recentchanges_ns_default 10 minute timeout)
  • # 3405
  • # 3401 .1 (10 minute timeout)
  • # 3395 .1 (test_recentchanges_default errors, however job timed out later after exceeded 120 min, and full error not shown
  • # 3394 .1 (test_recentchanges_default errors, however job timed out later after exceeded 120 min, and full error not shown)
  • # 3393 .1 (test_recentchanges_default 10 minute timeout)
  • # 3392 .1 (failed with InvalidTitle)
  • # 3391
  • # 3390 .1 (test_recentchanges_default and test_recentchanges_ns_default fail on InvalidTitle)
  • # 3389
  • # 3388
  • # 3387 .1 (failed with InvalidTitle)
  • # 3386 .1 (test_recentchanges_ns_default failed with InvalidTitle)
  • # 3385 .1 (test_recentchanges_ns_default failed with InvalidTitle)
  • # 3384 .1 (test_recentchanges_ns_default failed with InvalidTitle)
  • # 3383 .1 (test_recentchanges_ns_default failed with InvalidTitle)
  • # 3382 .1 (test_recentchanges_ns_default failed with InvalidTitle)
  • # 3381 .1 (test_recentchanges_ns_default failed with InvalidTitle)
  • # 3380
  • # 3379
  • # 3378
  • # 3377
  • # 3376 .1 (test_searchitem_language 10 minute timeout)
  • # 3373
  • # 3372
  • # 3371
  • # 3368 .1 (test_recentchanges_ns_default failed with InvalidTitle)

..

  • # 3358 .1 (test_recentchanges_ns_default failed with InvalidTitle)

...

  • # 3350 .1 (test_recentchanges_ns_default failed with InvalidTitle)

...

  • # 3346 .1 (test_recentchanges_ns_default failed with InvalidTitle)
  • # 3345 .1 (test_recentchanges_ns_default failed with InvalidTitle)
  • # 3344 .1 (looks good)

...

  • # 3340 (no, almost successful build)

Event Timeline

I suspect that df2c09aa75b2 introduced the InvalidTitle problem, and it may not have been fixed yet.
Part of this problem may be that travis-ci.org. mediawiki or the wiki data has changed, instead of it being a pywikibot code change.

Seems there is an autoblocked user in the stream. User class is able to handle it because it removes the preleading '#' but Page couldn't serve it. Maybe generators should always yield the right Page subclass.

Xqt triaged this task as High priority.

Change 284175 had a related patch set uploaded (by Xqt):
[bugfix] re-enable total parameter for RecentChangesPageGenerator

https://gerrit.wikimedia.org/r/284175

The timeout problem was cause by the very large amount of pages which where retrieved because the total parameter was not recognized. The patch should solve this part and maybe the other problem too when not all rc where retrieved anymore.

Change 284175 merged by jenkins-bot:
[bugfix] re-enable total parameter for RecentChangesPageGenerator

https://gerrit.wikimedia.org/r/284175