Page MenuHomePhabricator

pwb.py does not run scripts if user-config.py does not exist
Closed, ResolvedPublic

Description

if generate_family_file.py is invoked via pwb.py generate_family_file.py the framework tries to generate a user-config file first.


Version: core-(2.0)
Severity: normal

Details

Reference
bz72503
Related Gerrit Patches:
pywikibot/core : 2.0pwb fixups for generate_*
pywikibot/core : masterpwb fixups for generate_*

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:49 AM
bzimport set Reference to bz72503.
bzimport added a subscriber: Unknown Object (????).
jayvdb added a comment.EditedOct 25 2014, 4:31 PM

Please add more detailed steps to reproduce

$ python pwb.py generate_family_file.py
Usage: generate_family_file.py <url> <short name>
Example: generate_family_file.py https://www.mywiki.bogus/wiki/Main_Page mywiki
This will create the file families/mywiki_family.py
Please insert URL to wiki: http://glossary.reuters.com
Please insert a short name (eg: freeciv): reuters
Generating family file from http://glossary.reuters.com
*** WARNING: Api does not seem to be enabled on http://glossary.reuters.com

==================================
api url: http://glossary.reuters.com/api.php
MediaWiki version: 1.11.1
==================================

Determining other languages...
Loading wikis... 
  * en... in cache
Writing pywikibot/families/reuters_family.py... 
$ cat pywikibot/families/reuters_family.py
# -*- coding: utf-8 -*-
"""
This family file was auto-generated by $Id: 78e663f86b4d504a28387ecbf0c2e0e8e549cf4e $
Configuration parameters:
  url = http://glossary.reuters.com
  name = reuters

Please do not commit this to the Git repository!
"""

from pywikibot import family

class Family(family.Family):
    def __init__(self):
        family.Family.__init__(self)
        self.name = 'reuters'
        self.langs = {
            'en': 'glossary.reuters.com',
        }



    def scriptpath(self, code):
        return {
            'en': '',
        }[code]

    def version(self, code):
        return {
            'en': u'1.11.1',
        }[code]

C:\rewrite>pwb.py generate_family_file.py http://mysite.org/wiki/Main_Page mysit
e

NOTE: 'user-config.py' was not found! Please follow the prompts to create it: No user-config.py found in directory 'C:\rewrite'.
WARNING: Skipping loading of user-config.py. WARNING: Running on Windows and transliteration_target is not set. Please see https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:Pywikibot/Wi ndows WARNING: family and mylang are not set. Defaulting to family='test' and mylang='test'.

Your default user directory is "C:\rewrite"

WARNING: pywikibot.bot.inputChoice is deprecated, use input_choice instead. How to proceed? ([K]eep, [c]hange) k WARNING: pywikibot.bot.inputChoice is deprecated, use input_choice instead. Do you want to copy user files from an existing Pywikibot installation? ([y]es, [n]o) n WARNING: pywikibot.bot.inputChoice is deprecated, use input_choice instead. Create user-config.py file? Required for running bots. ([y]es, [N]o) n user-fixes.py already exists in the directory

Ah, so rather the issue is generate_family_file is not run when no user-config.py exists. What happens is the following:

  • pwb.py sees no user-config and thus invokes create_user_files.py
  • you don't want to create a user-config file yet, because you need the to-be-created family for that

The simplest way to fix this is to add a special case for generate_family_file.py in pwb.py, as it really *is* a special case. Allowing other scripts to continue would mean using test:test as site, for instance. Essentially it's our config singleton biting us again...

XZise added a comment.Oct 26 2014, 3:18 PM

(In reply to Merlijn van Deen from comment #3)

Essentially it's our config singleton biting us again...

The current version of 'generate_family_file' could be run without a user config (like generate_user_files). The problem is just that pwb.py doesn't know that. Although the question is how can we pwb.py let it know? We could have just a list of files which don't require a user config, but then we need to make sure it's kept in sync with the file name(s).

Or only load the (user) config as soon as the script requires it which would be the default. So in generate_family_file it's setting a environment variable to say "user config is optional" and only then it queries the code to load the config. The problem here is when a file doesn't need it but depends on a file which does. If the default (when nothing is done about it) is to require the user config (so we only need to change generate_family_file) and a file does request that it doesn't need a user config it sets that variable. Later it depends then on something which does require the user config but doesn't set the variable because by default it doesn't need to.

Both scenarios sound like I'm overthinking it so maybe someone has a better and easier idea.

But thinking about it, it'd be nice if generate_family_file knows which family file directories are configured to give the user the configured directories to choose from. But that would require an “load user-config.py if possible but ignore otherwise” option.

jayvdb added a comment.Nov 9 2014, 8:36 PM

Or generate_user_files could ask the user if they want to add a family , and invoke generate_family_file

jayvdb updated the task description. (Show Details)Apr 15 2015, 4:04 AM
jayvdb removed a project: Pywikibot-General.
jayvdb set Security to None.

Change 220108 had a related patch set uploaded (by John Vandenberg):
pwb fixups for generate_*

https://gerrit.wikimedia.org/r/220108

Change 220108 merged by jenkins-bot:
pwb fixups for generate_*

https://gerrit.wikimedia.org/r/220108

XZise closed this task as Resolved.Jun 24 2015, 9:52 AM
XZise assigned this task to jayvdb.

Change 223874 had a related patch set uploaded (by Merlijn van Deen):
pwb fixups for generate_*

https://gerrit.wikimedia.org/r/223874

Change 223874 merged by jenkins-bot:
pwb fixups for generate_*

https://gerrit.wikimedia.org/r/223874