Page MenuHomePhabricator

Page._quality has different formats for different pages
Closed, ResolvedPublic

Description

See https://phabricator.wikimedia.org/T198998.

page._quality can therefore be an int or a str.

This creates problems when filtering of quality level is used.

user@pc:~/python/core {master}$ python scripts/listpages.py -ql:3,4 -prefixindex:"Page:Popular Science Monthly Volume 49.djvu/99"
   1 Popular Science Monthly Volume 49.djvu/99
1 page(s) found
user@pc:~/python/core {master}$ python scripts/listpages.py -ql:3,4 -prefixindex:"Page:Popular Science Monthly Volume 49.djvu/100"
0 page(s) found

Details

Related Gerrit Patches:

Event Timeline

Mpaa created this task.Jul 6 2018, 8:51 PM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJul 6 2018, 8:51 PM

Change 444312 had a related patch set uploaded (by Mpaa; owner: Mpaa):
[pywikibot/core@master] Represent quality in ProofreadPage as int

https://gerrit.wikimedia.org/r/444312

Mpaa added a comment.Jul 6 2018, 8:53 PM
This comment was removed by Mpaa.
Mpaa updated the task description. (Show Details)Jul 6 2018, 8:55 PM
Mpaa renamed this task from Page_quality has different formats for different pages to Page._quality has different formats for different pages.Jul 6 2018, 9:05 PM
Dvorapa added a project: Pywikibot-tests.EditedJul 7 2018, 12:18 PM
Dvorapa updated the task description. (Show Details)
Dvorapa added a subscriber: Dvorapa.

Also it fails on AppVeyor and Travis

Dalba triaged this task as High priority.Jul 7 2018, 1:45 PM
Mpaa added a comment.EditedJul 7 2018, 5:03 PM

Also it fails on AppVeyor and Travis

I could not find a fail related to this.
It should fail like this:

F
======================================================================
FAIL: test_applicable_quality_level (tests.proofreadpage_tests.TestPageQuality)
Test Page.quality_level when applicable.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/python/core/tests/proofreadpage_tests.py", line 315, in test_applicable_quality_level
    self.assertEqual(page.quality_level, 0)
AssertionError: '0' != 0

due to 'quality': '0' here:

{'pageid': 435605, 'ns': 104, 'title': 'Page:Popular Science Monthly Volume 49.djvu/1', 'contentmodel': 'proofread-page', 'pagelanguage': 'en', 'pagelanguagehtmlcode': 'en', 'pagelanguagedir': 'ltr', 'touched': '2018-07-07T16:41:59Z', 'lastrevid': 7555835, 'length': 13, 'protection': [], 'restrictiontypes': ['edit', 'move'], 'proofread': {'quality': '0', 'quality_text': 'Without text'}}

For convenience, internally uses ints for quality value, so it should be converted to int as in the patch.

It fails like this:

======================================================================
FAIL: test_check_if_cached_enws (tests.proofreadpage_tests.TestIndexPageMappings)
Test if cache is checked and loaded properly on wikisource:en
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/wikimedia/pywikibot/tests/aspects.py", line 783, in wrapped_method
    func(self, key)
  File "/home/travis/build/wikimedia/pywikibot/tests/proofreadpage_tests.py", line 548, in test_check_if_cached
    fetched_label = index_page.get_label_from_page_number(num)
  File "/home/travis/build/wikimedia/pywikibot/pywikibot/proofreadpage.py", line 709, in wrapper
    self._get_page_mappings()
  File "/home/travis/build/wikimedia/pywikibot/pywikibot/proofreadpage.py", line 815, in _get_page_mappings
    % (page_cnt, title))
AssertionError: Page number 1 not recognised as page Page:Popular Science Monthly Volume 1.djvu/8.
======================================================================
FAIL: test_get_labels_enws (tests.proofreadpage_tests.TestIndexPageMappings)
Test IndexPage page get_label_from_* functions on wikisource:en
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/wikimedia/pywikibot/tests/aspects.py", line 783, in wrapped_method
    func(self, key)
  File "/home/travis/build/wikimedia/pywikibot/tests/proofreadpage_tests.py", line 575, in test_get_labels
    self.assertEqual(index_page.get_label_from_page_number(num), label)
  File "/home/travis/build/wikimedia/pywikibot/pywikibot/proofreadpage.py", line 709, in wrapper
    self._get_page_mappings()
  File "/home/travis/build/wikimedia/pywikibot/pywikibot/proofreadpage.py", line 815, in _get_page_mappings
    % (page_cnt, title))
AssertionError: Page number 1 not recognised as page Page:Popular Science Monthly Volume 1.djvu/8.
======================================================================
FAIL: test_get_page_and_number_enws (tests.proofreadpage_tests.TestIndexPageMappings)
Test IndexPage page get_page_number functions on wikisource:en
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/wikimedia/pywikibot/tests/aspects.py", line 783, in wrapped_method
    func(self, key)
  File "/home/travis/build/wikimedia/pywikibot/tests/proofreadpage_tests.py", line 592, in test_get_page_and_number
    self.assertEqual(index_page.get_page_number_from_label(label),
  File "/home/travis/build/wikimedia/pywikibot/pywikibot/proofreadpage.py", line 709, in wrapper
    self._get_page_mappings()
  File "/home/travis/build/wikimedia/pywikibot/pywikibot/proofreadpage.py", line 815, in _get_page_mappings
    % (page_cnt, title))
AssertionError: Page number 1 not recognised as page Page:Popular Science Monthly Volume 1.djvu/8.
======================================================================
FAIL: test_num_pages_enws (tests.proofreadpage_tests.TestIndexPageMappings)
Test num_pages property on wikisource:en
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/wikimedia/pywikibot/tests/aspects.py", line 783, in wrapped_method
    func(self, key)
  File "/home/travis/build/wikimedia/pywikibot/tests/proofreadpage_tests.py", line 563, in test_num_pages
    self.assertEqual(index_page.num_pages, self.sites[key]['num_pages'])
  File "/home/travis/build/wikimedia/pywikibot/pywikibot/proofreadpage.py", line 709, in wrapper
    self._get_page_mappings()
  File "/home/travis/build/wikimedia/pywikibot/pywikibot/proofreadpage.py", line 815, in _get_page_mappings
    % (page_cnt, title))
AssertionError: Page number 1 not recognised as page Page:Popular Science Monthly Volume 1.djvu/8.
======================================================================
FAIL: test_page_gen_enws (tests.proofreadpage_tests.TestIndexPageMappings)
Test Index page generator on wikisource:en
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/wikimedia/pywikibot/tests/aspects.py", line 783, in wrapped_method
    func(self, key)
  File "/home/travis/build/wikimedia/pywikibot/tests/proofreadpage_tests.py", line 633, in test_page_gen
    self.assertRaises(ValueError, index_page.page_gen, -1, 2)
AssertionError: Page number 1 not recognised as page Page:Popular Science Monthly Volume 1.djvu/8.
Dvorapa added a comment.EditedJul 7 2018, 6:34 PM

Is it the same issue? Or are they caused by the same code part/API point at least? (It seemed to me so)

Mpaa added a comment.Jul 7 2018, 8:25 PM

Is it the same issue? Or are they caused by the same code part/API point at least? (It seemed to me so)

The root cause is T114318.

Is it the same issue? Or are they caused by the same code part/API point at least? (It seemed to me so)

The root cause is T114318.

I see, maybe they should be marked as parent/subtask?

Mpaa added a comment.Jul 7 2018, 8:54 PM

In case I was unclear, T114318 is the root cause for the failure you showed.

I see, they are different

Dalba added a subscriber: Dalba.EditedJul 8 2018, 5:47 AM

Most CI jobs are failing:
https://ci.appveyor.com/project/Ladsgroup/pywikibot-g4xqx/build/1.0.544/job/okvq8tnmesf72b87#L4742
https://travis-ci.org/wikimedia/pywikibot/jobs/401363307#L5023

FAIL: test_applicable_quality_level (tests.proofreadpage_tests.TestPageQuality)
Test Page.quality_level when applicable.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\projects\pywikibot-g4xqx\tests\proofreadpage_tests.py", line 316, in test_applicable_quality_level
    self.assertEqual(page.quality_level, 0)
AssertionError: '0' != 0

Change 444312 merged by jenkins-bot:
[pywikibot/core@master] Represent quality in ProofreadPage as int

https://gerrit.wikimedia.org/r/444312

Dalba closed this task as Resolved.Jul 8 2018, 10:30 AM
Dalba assigned this task to Mpaa.