Page MenuHomePhabricator

Add unit tests for YearPageGenerator and DayPageGenerator
Closed, ResolvedPublic

Description

Tests are needed for YearPageGenerator and DayPageGenerator - these should be subclasses of DefaultSiteTestCase , and run on all travis build sites (test.wikidata, en.wp, ar.wp and fr.wikt - https://travis-ci.org/wikimedia/pywikibot-core/builds), and also be tested on the production wikidata site.

It is very likely that these generators will not work as expected on non-Wikipedia sites, as https://fr.wiktionary.org/wiki/2010 and https://en.wiktionary.org/wiki/2010 do not exist, and Wikidata is very different and the date module has not been updated since Wikidata was created. These tests should identify these bugs, so that they can be fixed.

Event Timeline

jayvdb raised the priority of this task from to Needs Triage.
jayvdb updated the task description. (Show Details)
jayvdb added subscribers: Aklapper, Unknown Object (MLST), jayvdb.

I wrote a basic test (not yet submitted to Gerrit) for the year page generator:

class TestYearPageGenerator(DefaultSiteTestCase):

    """Test the year page generator"""

    def test_basic(self):
        site = self.get_site()
        start = 42
        end = 2026

        i = 0
        for page in pagegenerators.YearPageGenerator(start, end, site):
            self.assertIsInstance(page, pywikibot.Page)
            self.assertEqual(date.formatYear(site.lang, start+i), page.title(asLink=False));
            i += 1
        self.assertEqual(start+i-1, end)

It checks that each Page instance was generated and that it has the correct title, and also that the correct number of pages were generated. I have tested it on a few of the mentioned non-Wikipedia sites without any issues, however my test does not even issue any API requests (I don't even check .exists(), as that takes a while for all of the years. Should I reduce the number of years I test and also test weather .exists returns a bool, as other tests do?).

This bug report suggested that it was likely these generators would not work as expected, but as I am not very familiar with pywikibot, I have no sense of how these might fail, or what else I need to test.

I also wrote a simplistic test (also not submitted to Gerrit) for the day page generator:

class TestDayPageGenerator(DefaultSiteTestCase):

    """Test the day page generator"""

    def test_basic(self):
        site = self.get_site()
        fd = date.FormatDate(site)
        startMonth = 1
        endMonth = 12

        gen = pagegenerators.DayPageGenerator(startMonth, endMonth, site)

        expected = []
        for month in range(startMonth, endMonth + 1):
            for day in range(1, date.getNumberOfDaysInMonth(month) + 1):
                expected.append(fd(month, day))

        self.assertPageTitlesEqual(gen, expected)

        for page in pagegenerators.DayPageGenerator(startMonth, endMonth, site):
            self.assertIsInstance(page, pywikibot.Page)
            self.assertIsInstance(page.exists(), bool) # this takes a long while

Any advice is welcome.

Oh, and Happy New Year as far as UTC time goes.

After some further testing, I discovered T85645 on wikidata. However, it sounded like it was anticipated that I would find many more on other non-Wikipedia sites.

Change 182405 had a related patch set uploaded (by Unicornisaurous):
Add unit tests for YearPageGenerator and DayPageGenerator

https://gerrit.wikimedia.org/r/182405

Patch-For-Review

Change 182405 merged by jenkins-bot:
Add unit tests for YearPageGenerator and DayPageGenerator

https://gerrit.wikimedia.org/r/182405

This broke the builds

https://travis-ci.org/wikimedia/pywikibot-core/builds

(we'll need to revert if a fix cant be found quickly)

Change 182428 had a related patch set uploaded (by Unicornisaurous):
Fix TestYearPageGenerator when date formats are missing

https://gerrit.wikimedia.org/r/182428

Patch-For-Review

I added a temporary fix which will skip the test if the language used is missing from the formats dictionary for 'YearBC'. I have tested this with language 'ar' on 'wikipedia' (same as travis ci) so it should fix it.

I will search for an existing bug for missing formats like these and create one if none exist.

Change 182428 merged by jenkins-bot:
Fix TestYearPageGenerator when date formats are missing

https://gerrit.wikimedia.org/r/182428