Page MenuHomePhabricator
Create Task
Maniphest T233438

script_tests collector function should be validated
Open, LowPublic
Actions

Description

This is the code to collect the scripts

tests = (['test__login']
         + ['test_' + name
             for name in sorted(script_list)
             if name != 'login'
             and name not in failed_dep_script_set
             and name not in unrunnable_script_set
             and (enable_autorun_tests or name not in auto_run_script_list)
            ])

test_list += ['tests.script_tests.TestScriptSimulate.' + name
              for name in tests]

It's not clear why TestScriptSimulate.test_flickrripper is not in the final test_list
but the test is made in Travis.

Here are results of lists and sets used by the collector funtion of script_tests

C:\pwb\GIT\core>pwb -v script_tests
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:06:47) [MSC v.1914 32 bit (Inte
l)]
tests: max_retries reduced from 15 to 1
flickrripper depends on flickrapi, which isn't available
imageharvest depends on bs4, which isn't available
isbn depends on python-stdnum, which isn't available
match_images depends on PIL.ImageTk, which isn't available
states_redirect depends on pycountry, which isn't available
patrol depends on mwparserfromhell>=0.3.3, which isn't available
weblinkchecker depends on memento_client>=0.5.1,!=0.6.0, which isn't available
Skipping execution of unrunnable scripts:
  {'version'}
Skipping execution of auto-run scripts (set PYWIKIBOT_TEST_AUTORUN=1 to enable):

  ['blockpageschecker', 'casechecker', 'catall', 'category_redirect', 'cfd', 'ch
eckimages', 'clean_sandbox', 'disambredir', 'followlive', 'imagerecat', 'login',
 'lonelypages', 'misspelling', 'revertbot', 'noreferences', 'nowcommons', 'patro
l', 'shell', 'standardize_interwiki', 'states_redirect', 'unusedfiles', 'upload'
, 'watchlist', 'welcome']

'### script_list ###'
['login',
 'add_text',
 'archivebot',
 'basic',
 'blockpageschecker',
 'capitalize_redirects',
 'casechecker',
 'catall',
 'category',
 'category_redirect',
 'cfd',
 'change_pagelang',
 'checkimages',
 'claimit',
 'clean_sandbox',
 'commonscat',
 'commons_link',
 'coordinate_import',
 'cosmetic_changes',
 'create_categories',
 'data_ingestion',
 'delete',
 'disambredir',
 'djvutext',
 'editarticle',
 'fixing_redirects',
 'flickrripper',
 'followlive',
 'freebasemappingupload',
 'harvest_template',
 'illustrate_wikidata',
 'image',
 'imagecopy',
 'imagecopy_self',
 'imageharvest',
 'imagerecat',
 'imagetransfer',
 'imageuncat',
 'interwiki - Kopie',
 'interwiki',
 'interwikidata',
 'isbn',
 'listpages',
 'lonelypages',
 'makecat - Kopie',
 'makecat',
 'match_images',
 'misspelling',
 'movepages',
 'ndashredir',
 'newitem',
 'noreferences',
 'nowcommons',
 'pagefromfile',
 'patrol',
 'patrol_rewrite',
 'piper',
 'protect',
 'redirect',
 'reflinks',
 'replace',
 'replicate_wiki',
 'revertbot - Kopie',
 'revertbot',
 'selflink',
 'shell',
 'solve_disambiguation',
 'spamremove',
 'standardize_interwiki',
 'states_redirect',
 'subster - Kopie',
 'subster-compat',
 'surnames_redirects',
 'table2wiki',
 'template',
 'templatecount',
 'template_count - Kopie',
 'touch',
 'transferbot',
 'unlink',
 'unusedfiles',
 'unused_files - Kopie',
 'upload',
 'version',
 'watchlist',
 'weblinkchecker',
 'welcome',
 'wikisourcetext']

'### failed_dep_script_set ###'
{'flickrripper',
 'imageharvest',
 'isbn',
 'match_images',
 'patrol',
 'states_redirect',
 'weblinkchecker'}

'### unrunnable_script_set ###'
{'version'}

### enable_autorun_tests: False

'### auto_run_script_list ###'
['blockpageschecker',
 'casechecker',
 'catall',
 'category_redirect',
 'cfd',
 'checkimages',
 'clean_sandbox',
 'disambredir',
 'followlive',
 'imagerecat',
 'login',
 'lonelypages',
 'misspelling',
 'revertbot',
 'noreferences',
 'nowcommons',
 'patrol',
 'shell',
 'standardize_interwiki',
 'states_redirect',
 'unusedfiles',
 'upload',
 'watchlist',
 'welcome']

'### test_list ###'
['tests.script_tests.TestScriptHelp.test__login',
 'tests.script_tests.TestScriptHelp.test_add_text',
 'tests.script_tests.TestScriptHelp.test_archivebot',
 'tests.script_tests.TestScriptHelp.test_basic',
 'tests.script_tests.TestScriptHelp.test_blockpageschecker',
 'tests.script_tests.TestScriptHelp.test_capitalize_redirects',
 'tests.script_tests.TestScriptHelp.test_casechecker',
 'tests.script_tests.TestScriptHelp.test_catall',
 'tests.script_tests.TestScriptHelp.test_category',
 'tests.script_tests.TestScriptHelp.test_category_redirect',
 'tests.script_tests.TestScriptHelp.test_cfd',
 'tests.script_tests.TestScriptHelp.test_change_pagelang',
 'tests.script_tests.TestScriptHelp.test_checkimages',
 'tests.script_tests.TestScriptHelp.test_claimit',
 'tests.script_tests.TestScriptHelp.test_clean_sandbox',
 'tests.script_tests.TestScriptHelp.test_commons_link',
 'tests.script_tests.TestScriptHelp.test_commonscat',
 'tests.script_tests.TestScriptHelp.test_coordinate_import',
 'tests.script_tests.TestScriptHelp.test_cosmetic_changes',
 'tests.script_tests.TestScriptHelp.test_create_categories',
 'tests.script_tests.TestScriptHelp.test_data_ingestion',
 'tests.script_tests.TestScriptHelp.test_delete',
 'tests.script_tests.TestScriptHelp.test_disambredir',
 'tests.script_tests.TestScriptHelp.test_djvutext',
 'tests.script_tests.TestScriptHelp.test_editarticle',
 'tests.script_tests.TestScriptHelp.test_fixing_redirects',
 'tests.script_tests.TestScriptHelp.test_flickrripper',
 'tests.script_tests.TestScriptHelp.test_followlive',
 'tests.script_tests.TestScriptHelp.test_freebasemappingupload',
 'tests.script_tests.TestScriptHelp.test_harvest_template',
 'tests.script_tests.TestScriptHelp.test_illustrate_wikidata',
 'tests.script_tests.TestScriptHelp.test_image',
 'tests.script_tests.TestScriptHelp.test_imagecopy',
 'tests.script_tests.TestScriptHelp.test_imagecopy_self',
 'tests.script_tests.TestScriptHelp.test_imageharvest',
 'tests.script_tests.TestScriptHelp.test_imagerecat',
 'tests.script_tests.TestScriptHelp.test_imagetransfer',
 'tests.script_tests.TestScriptHelp.test_imageuncat',
 'tests.script_tests.TestScriptHelp.test_interwiki',
 'tests.script_tests.TestScriptHelp.test_interwiki - Kopie',
 'tests.script_tests.TestScriptHelp.test_interwikidata',
 'tests.script_tests.TestScriptHelp.test_isbn',
 'tests.script_tests.TestScriptHelp.test_listpages',
 'tests.script_tests.TestScriptHelp.test_lonelypages',
 'tests.script_tests.TestScriptHelp.test_makecat',
 'tests.script_tests.TestScriptHelp.test_makecat - Kopie',
 'tests.script_tests.TestScriptHelp.test_match_images',
 'tests.script_tests.TestScriptHelp.test_misspelling',
 'tests.script_tests.TestScriptHelp.test_movepages',
 'tests.script_tests.TestScriptHelp.test_ndashredir',
 'tests.script_tests.TestScriptHelp.test_newitem',
 'tests.script_tests.TestScriptHelp.test_noreferences',
 'tests.script_tests.TestScriptHelp.test_nowcommons',
 'tests.script_tests.TestScriptHelp.test_pagefromfile',
 'tests.script_tests.TestScriptHelp.test_patrol',
 'tests.script_tests.TestScriptHelp.test_patrol_rewrite',
 'tests.script_tests.TestScriptHelp.test_piper',
 'tests.script_tests.TestScriptHelp.test_protect',
 'tests.script_tests.TestScriptHelp.test_redirect',
 'tests.script_tests.TestScriptHelp.test_reflinks',
 'tests.script_tests.TestScriptHelp.test_replace',
 'tests.script_tests.TestScriptHelp.test_replicate_wiki',
 'tests.script_tests.TestScriptHelp.test_revertbot',
 'tests.script_tests.TestScriptHelp.test_revertbot - Kopie',
 'tests.script_tests.TestScriptHelp.test_selflink',
 'tests.script_tests.TestScriptHelp.test_shell',
 'tests.script_tests.TestScriptHelp.test_solve_disambiguation',
 'tests.script_tests.TestScriptHelp.test_spamremove',
 'tests.script_tests.TestScriptHelp.test_standardize_interwiki',
 'tests.script_tests.TestScriptHelp.test_states_redirect',
 'tests.script_tests.TestScriptHelp.test_subster - Kopie',
 'tests.script_tests.TestScriptHelp.test_subster-compat',
 'tests.script_tests.TestScriptHelp.test_surnames_redirects',
 'tests.script_tests.TestScriptHelp.test_table2wiki',
 'tests.script_tests.TestScriptHelp.test_template',
 'tests.script_tests.TestScriptHelp.test_template_count - Kopie',
 'tests.script_tests.TestScriptHelp.test_templatecount',
 'tests.script_tests.TestScriptHelp.test_touch',
 'tests.script_tests.TestScriptHelp.test_transferbot',
 'tests.script_tests.TestScriptHelp.test_unlink',
 'tests.script_tests.TestScriptHelp.test_unused_files - Kopie',
 'tests.script_tests.TestScriptHelp.test_unusedfiles',
 'tests.script_tests.TestScriptHelp.test_upload',
 'tests.script_tests.TestScriptHelp.test_watchlist',
 'tests.script_tests.TestScriptHelp.test_weblinkchecker',
 'tests.script_tests.TestScriptHelp.test_welcome',
 'tests.script_tests.TestScriptHelp.test_wikisourcetext',
 'tests.script_tests.TestScriptSimulate.test__login',
 'tests.script_tests.TestScriptSimulate.test_add_text',
 'tests.script_tests.TestScriptSimulate.test_archivebot',
 'tests.script_tests.TestScriptSimulate.test_basic',
 'tests.script_tests.TestScriptSimulate.test_capitalize_redirects',
 'tests.script_tests.TestScriptSimulate.test_category',
 'tests.script_tests.TestScriptSimulate.test_change_pagelang',
 'tests.script_tests.TestScriptSimulate.test_claimit',
 'tests.script_tests.TestScriptSimulate.test_commons_link',
 'tests.script_tests.TestScriptSimulate.test_commonscat',
 'tests.script_tests.TestScriptSimulate.test_coordinate_import',
 'tests.script_tests.TestScriptSimulate.test_cosmetic_changes',
 'tests.script_tests.TestScriptSimulate.test_create_categories',
 'tests.script_tests.TestScriptSimulate.test_data_ingestion',
 'tests.script_tests.TestScriptSimulate.test_delete',
 'tests.script_tests.TestScriptSimulate.test_djvutext',
 'tests.script_tests.TestScriptSimulate.test_editarticle',
 'tests.script_tests.TestScriptSimulate.test_fixing_redirects',
 'tests.script_tests.TestScriptSimulate.test_freebasemappingupload',
 'tests.script_tests.TestScriptSimulate.test_harvest_template',
 'tests.script_tests.TestScriptSimulate.test_illustrate_wikidata',
 'tests.script_tests.TestScriptSimulate.test_image',
 'tests.script_tests.TestScriptSimulate.test_imagecopy',
 'tests.script_tests.TestScriptSimulate.test_imagecopy_self',
 'tests.script_tests.TestScriptSimulate.test_imagetransfer',
 'tests.script_tests.TestScriptSimulate.test_imageuncat',
 'tests.script_tests.TestScriptSimulate.test_interwiki',
 'tests.script_tests.TestScriptSimulate.test_interwiki - Kopie',
 'tests.script_tests.TestScriptSimulate.test_interwikidata',
 'tests.script_tests.TestScriptSimulate.test_listpages',
 'tests.script_tests.TestScriptSimulate.test_makecat',
 'tests.script_tests.TestScriptSimulate.test_makecat - Kopie',
 'tests.script_tests.TestScriptSimulate.test_movepages',
 'tests.script_tests.TestScriptSimulate.test_ndashredir',
 'tests.script_tests.TestScriptSimulate.test_newitem',
 'tests.script_tests.TestScriptSimulate.test_pagefromfile',
 'tests.script_tests.TestScriptSimulate.test_patrol_rewrite',
 'tests.script_tests.TestScriptSimulate.test_piper',
 'tests.script_tests.TestScriptSimulate.test_protect',
 'tests.script_tests.TestScriptSimulate.test_redirect',
 'tests.script_tests.TestScriptSimulate.test_reflinks',
 'tests.script_tests.TestScriptSimulate.test_replace',
 'tests.script_tests.TestScriptSimulate.test_replicate_wiki',
 'tests.script_tests.TestScriptSimulate.test_revertbot - Kopie',
 'tests.script_tests.TestScriptSimulate.test_selflink',
 'tests.script_tests.TestScriptSimulate.test_solve_disambiguation',
 'tests.script_tests.TestScriptSimulate.test_spamremove',
 'tests.script_tests.TestScriptSimulate.test_subster - Kopie',
 'tests.script_tests.TestScriptSimulate.test_subster-compat',
 'tests.script_tests.TestScriptSimulate.test_surnames_redirects',
 'tests.script_tests.TestScriptSimulate.test_table2wiki',
 'tests.script_tests.TestScriptSimulate.test_template',
 'tests.script_tests.TestScriptSimulate.test_template_count - Kopie',
 'tests.script_tests.TestScriptSimulate.test_templatecount',
 'tests.script_tests.TestScriptSimulate.test_touch',
 'tests.script_tests.TestScriptSimulate.test_transferbot',
 'tests.script_tests.TestScriptSimulate.test_unlink',
 'tests.script_tests.TestScriptSimulate.test_unused_files - Kopie',
 'tests.script_tests.TestScriptSimulate.test_wikisourcetext']
flickrripper depends on flickrapi, which isn't available
imageharvest depends on bs4, which isn't available
isbn depends on python-stdnum, which isn't available
match_images depends on PIL.ImageTk, which isn't available
states_redirect depends on pycountry, which isn't available
patrol depends on mwparserfromhell>=0.3.3, which isn't available
weblinkchecker depends on memento_client>=0.5.1,!=0.6.0, which isn't available

C:\pwb\GIT\core>

Details

SubjectRepoBranchLines +/-
[cleanup] Remove load_test and collector from script_tests.pypywikibot/coremaster+0 -50
[bufix] Skip TestScriptSimulate test with open dependenciespywikibot/coremaster+36 -43
Customize query in gerrit

Related Objects
Search...

Event Timeline

Xqt created this task.Sep 20 2019, 4:22 PM
Restricted Application added a project: Pywikibot. · View Herald TranscriptSep 20 2019, 4:22 PM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald Transcript
Xqt updated the task description. (Show Details)Sep 20 2019, 4:28 PM
Xqt updated the task description. (Show Details)Sep 20 2019, 4:33 PM
Dvorapa subscribed.Sep 20 2019, 5:11 PM

I did some work on this issue here: https://github.com/dvorapa/pywikibot/commits/dvorapa-test

Xqt added a comment.Sep 21 2019, 2:05 PM
In T233438#5510005, @Dvorapa wrote:

I did some work on this issue here: https://github.com/dvorapa/pywikibot/commits/dvorapa-test

Yes I guess this test script must be rewriten because it is not easy to understand.

Xqt triaged this task as Low priority.Sep 21 2019, 2:05 PM
Dvorapa added a comment.Sep 22 2019, 12:16 PM

Agree

Xqt added a subtask: T248297: some TestScriptSimulate tests are failing due to missing dependency package.Mar 23 2020, 7:56 AM
Xqt claimed this task.Mar 23 2020, 8:24 AM
gerritbot added a comment.Mar 23 2020, 4:11 PM

Change 582830 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [bufix] Skip TestScriptSimulate test with open dependencies

https://gerrit.wikimedia.org/r/582830

gerritbot added a project: Patch-For-Review.Mar 23 2020, 4:11 PM
gerritbot added a comment.Mar 23 2020, 10:34 PM

Change 582830 merged by jenkins-bot:
[pywikibot/core@master] [bufix] Skip TestScriptSimulate test with open dependencies

https://gerrit.wikimedia.org/r/582830

Xqt mentioned this in rPWBC9ed10fda7de8: [bufix] Skip TestScriptSimulate test with open dependencies.Mar 23 2020, 10:36 PM
Xqt closed subtask T248297: some TestScriptSimulate tests are failing due to missing dependency package as Resolved.Mar 24 2020, 6:53 AM
Xqt added a comment.Mar 24 2020, 6:56 AM

btw for what is this collector good for? Tests are made without it.

Xqt removed Xqt as the assignee of this task.Mar 24 2020, 6:57 AM
Dvorapa added a comment.Mar 24 2020, 7:51 AM

Without? Why do you think so?

Xqt added a comment.Mar 24 2020, 2:53 PM
In T233438#5994095, @Dvorapa wrote:

Without? Why do you think so?

I removed the load_test function and collector and run the test script.

Dvorapa added a comment.EditedMar 24 2020, 3:14 PM
In T233438#5994981, @Xqt wrote:
In T233438#5994095, @Dvorapa wrote:

Without? Why do you think so?

I removed the load_test function and collector and run the test script.

Well unittest has its default load_test and collector, so it is not mandatory. But we want to exclude some tests if they don't have dependencies, so we need to write our own load_test and collector. I don't know, how we exclude these in pytest?

Dvorapa added a comment.Mar 24 2020, 4:21 PM

pytest uses collect_ignore list in conftest.py per docs

Xqt added a comment.Mar 24 2020, 4:34 PM
In T233438#5995477, @Dvorapa wrote:

pytest uses collect_ignore list in conftest.py per docs

I think skipping the tests like I did with https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/582830/ should be enough and doesn't need any additional workaround for different test environments.

Xqt removed a project: Patch-For-Review.Mar 24 2020, 4:35 PM
Dvorapa added a comment.Mar 24 2020, 4:45 PM

OK, let's drop load_tests and collector from script_tests.py, where unittest.skip can be used instead.

Note: We can not drop these from tests/__init__.py as default unittest load_tests and collector expects test_pagegenerators.py instead of pagegenerators_tests.py.

Xqt added a comment.Mar 25 2020, 11:33 AM

There is a little difference: With load_test the log looks like:

test__login (tests.script_tests.TestScriptHelp)
Test running login -help. ... ok
test_add_text (tests.script_tests.TestScriptHelp)
Test running add_text -help. ... ok
test_archivebot (tests.script_tests.TestScriptHelp)
Test running archivebot -help. ... ok

without it it Looks like

test__login (__main__.TestScriptHelp)
Test running login -help. ... ok
test_add_text (__main__.TestScriptHelp)
Test running add_text -help. ... ok
test_archivebot (__main__.TestScriptHelp)
Test running archivebot -help. ... ok

But does this care?

gerritbot added a comment.Mar 25 2020, 11:44 AM

Change 583312 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [cleanup] Remove load_test and collector from script_tests.py

https://gerrit.wikimedia.org/r/583312

gerritbot added a project: Patch-For-Review.Mar 25 2020, 11:44 AM
Dvorapa added a comment.Mar 25 2020, 11:54 AM

There is another difference: If there is a login issue (can not login to **:wikipedia), with load_tests it skips only one script test, without it kips the whole TestScript(Simulate,Help) class.

Xqt renamed this task from script_tests' collector function should be validated to script_tests collector function should be validated.Mar 25 2020, 2:50 PM
Dvorapa moved this task from Backlog to Framework on the Pywikibot-tests board.Apr 3 2020, 4:14 PM
Pppery edited projects, added Patch-Needs-Improvement; removed Patch-For-Review.Mar 28 2023, 12:28 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL