Page MenuHomePhabricator

Trying to get pywikibot running by following documentation fails
Closed, ResolvedPublic

Description

user-config.py

# -*- coding: utf-8 -*-
import os
family = 'royalfamily'
register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family)
mylang = 'en'
usernames[family]['en'] = u'wf'
console_encoding = 'utf-8'
password_file = "wf-password.py"

migrate.py

#!/usr/bin/python
# -*- coding: utf-8 -*-
import pywikibot
mysite = pywikibot.Site()
print(mysite.sitename)
python3 migrate.py 
Traceback (most recent call last):
  File "migrate.py", line 4, in <module>
    mysite = pywikibot.Site()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/__init__.py", line 1237, in Site
    fam = Family.load(fam)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/tools/__init__.py", line 1738, in wrapper
    return obj(*__args, **__kw)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/family.py", line 1008, in load
    raise UnknownFamily('Family %s does not exist' % fam)
pywikibot.exceptions.UnknownFamily: Family royalfamily does not exist
CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.UnknownFamily'>

alternatively try

family_files[family] = 'http://royal-family.bitplan.com/'

to get

WARNING: /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/site.py:774: UserWarning: Global configuration variable "mylang" changed to "royalfamily" while instantiating site royalfamily:royalfamily
  % (self.__code, self), UserWarning)

WARNING: migrate.py:4: UserWarning: Site royalfamily:royalfamily instantiated using different code "en"
  mysite = pywikibot.Site()

royalfamily:royalfamily

none of this makes sense to me and i commented on the other phabricator tasks with similar issues that the documentation and the experience seems to be inconsistent.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 28 2019, 6:07 PM
Seppl2013 updated the task description. (Show Details)Sep 28 2019, 6:08 PM
Seppl2013 updated the task description. (Show Details)
Restricted Application added a subscriber: pywikibot-bugs-list. · View Herald TranscriptSep 28 2019, 6:09 PM
Aklapper changed the task status from Open to Stalled.Sep 28 2019, 9:54 PM

Hi, please always explain which specific documentation you are following. Do you refer to https://www.mediawiki.org/wiki/Manual:Pywikibot/Use_on_third-party_wikis ?

alternatively try
family_files[family] = 'http://royal-family.bitplan.com/'

Try where? In user-config.py? Please be specific, thanks. :)

to get

There are some warnings, but you do not explain what the actual problem is. Please see and follow https://www.mediawiki.org/wiki/How_to_report_a_bug

Xqt added a subscriber: Xqt.Sep 29 2019, 1:52 AM

@Seppl2013: Probably you should use generate_family_files.py to create you private family file.

thx for the comments. The actual problem is that the warnings do not make sense. They refer to something independent like Global configuration variable "mylang" and are therefor confusing for newbies. Yes https://www.mediawiki.org/wiki/Manual:Pywikibot/Use_on_third-party_wikis is the documentation I tried for this. Yes the alternative statement is in the same user-config.py file and commented out/in for the two attempts i made. I do not know how to use generate_family_files.py. The documentation says:

The generate_family_file.py in the root directory

Obviously it's not referrring to the root directory of my MacOS installation. But it looks like i am expected to search for this root directory. So a google search shows me https://phabricator.wikimedia.org/T153891

So I am expecting it will crash if i try it ... since that task is open since December 2016 ...

Seppl2013 added a comment.EditedSep 29 2019, 1:14 PM

Ok I am continuing my search for the root directory in which to find generate_family.py - at https://www.mediawiki.org/wiki/Manual:Pywikibot/Scripts it's a red link ...
I do find it at https://phabricator.wikimedia.org/diffusion/PWBC/repository/master/ but i didn't install things from phabricator. I used a
requirements.txt

# 2019-09-28
Pywikibot

and did a

$pip install -r requirements.txt

making sure the right pip is used in this case the one for python3 ...

pip show Pywikibot

tells me a long path in wich I try to find the generate_family_file.py but I get an empty result on:

find /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages -name "generate_fam*"
Dvorapa added a subscriber: Dvorapa.EditedSep 29 2019, 1:20 PM

It means the root directory of your Pywikibot installation

If you used pip to install Pywikibot, then this file generate_family_file.py is not included in your Pywikibot installation and you have to create your family file by hand in the families directory

I tried:

register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+".py")

in
user-config.py

# -*- coding: utf-8 -*-
import os
family = 'royalfamily'
register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+".py")
mylang = 'en'
usernames[family]['en'] = u'wf'
console_encoding = 'utf-8'
password_file = "wf-password.py"
#family_files[family] = 'http://royal-family.bitplan.com/'
#register_families_folder())
#use_api_login = True

and

cat royalfamily_family.py 
from pywikibot import family

class Family(family.Family):
    name = 'royalfamily'
    langs = {
        'en': 'http://royal-family.bitplan.com/'
    }

with the result:

python3 migrate.py 
Traceback (most recent call last):
  File "migrate.py", line 4, in <module>
    mysite = pywikibot.Site()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/__init__.py", line 1237, in Site
    fam = Family.load(fam)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/tools/__init__.py", line 1738, in wrapper
    return obj(*__args, **__kw)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/family.py", line 1008, in load
    raise UnknownFamily('Family %s does not exist' % fam)
pywikibot.exceptions.UnknownFamily: Family royalfamily does not exist
CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.UnknownFamily'>
Xqt changed the task status from Stalled to Open.Sep 30 2019, 10:25 AM

Issue is in your family file detection, because os.path.expanduser("~")+'/.pywikibot/families'+"/"+family+".py" gives royalfamily.py, not royalfamily_family.py

I assume your folder structure is like this (in your user folder, often abbreviated as ~):

.pywikibot
-- user_config.py
-- families
   -- royalfamily_family.py

You should use os.path.expanduser('~') + '/.pywikibot/families/' + family + '_family.py' instead. See:

$ python
Python 3.7.4 (default, Jul 16 2019, 07:12:58) 
[GCC 9.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import pywikibot  // no need to import in user_config.py
>>> from pywikibot import config2  // no need to import in user_config.py
>>> family = 'royalfamily'
>>> mylang = 'en'
>>> config2.register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+".py")
>>> pywikibot.Site(mylang, family)
Traceback (most recent call last):
  File "/home/pavel/.local/lib/python3.7/site-packages/pywikibot/family.py", line 1017, in load
    mod = import_module(splitext(basename(family_file))[0])
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'royalfamily'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pavel/.local/lib/python3.7/site-packages/pywikibot/__init__.py", line 1237, in Site
    fam = Family.load(fam)
  File "/home/pavel/.local/lib/python3.7/site-packages/pywikibot/tools/__init__.py", line 1738, in wrapper
    return obj(*__args, **__kw)
  File "/home/pavel/.local/lib/python3.7/site-packages/pywikibot/family.py", line 1019, in load
    raise UnknownFamily('Family %s does not exist' % fam)
pywikibot.exceptions.UnknownFamily: Family royalfamily does not exist
>>> config2.register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+"_family.py")
>>> pywikibot.Site(mylang, family)
APISite("en", "royalfamily")
Dvorapa changed the task status from Open to Stalled.Sep 30 2019, 11:19 AM

Was it helpful? Have you succeeded? Can we close this?

Seppl2013 added a comment.EditedSep 30 2019, 11:47 AM

Indeed I get the same "no module named royalfamily" error as you do with:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import os
import pywikibot  # no need to import in user_config.py
from pywikibot import config2  # no need to import in user_config.py
family = 'royalfamily'
mylang = 'en'
config2.register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+".py")
mysite=pywikibot.Site(mylang, family)
print(mysite.sitename)

So I improved it to:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import os
import pywikibot  # no need to import in user_config.py
from pywikibot import config2  # no need to import in user_config.py
family = 'royalfamily'
mylang = 'en'
familyfile=os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+"_family.py"
if not os.path.isfile(familyfile):
  print ("family file %s is missing" % (familyfile))
else:
  print ("will try to use family file %s" % (familyfile))
config2.register_family_file(family, familyfile)
mysite=pywikibot.Site(mylang, family)
print(mysite.sitename)

With the expected result

python3 migrate.py 
will try to use family file /Users/wf/.pywikibot/families/royalfamily_family.py
royalfamily:en

I have added the example code to https://www.mediawiki.org/wiki/Manual:Pywikibot/Use_on_third-party_wikis#Configuring_custom_families_folder and recommend to close this issue.

Thanks a lot for the swift reactions and in depth discussion.

Sorry - the next issue is coming up:

WARNING: /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/family.py:71: _NotImplementedWarning: royalfamily_family.new__.init__ is deprecated. cls.instance.init()

Which looks like the syntax:

# -*- coding: utf-8  -*-
from pywikibot import family

class Family(family.Family):
    def __init__(self):
        self.name = 'royalfamily'
        self.langs = {
            'en': 'http://royal-family.bitplan.com/'
        }
        
    def scriptpath(self, code):
         return ''

is not o.k. although I just changed the doc page assuming the examples in https://www.mediawiki.org/wiki/Manual:Pywikibot/Third-party_Wiki_Quick_Start would be o.k.

Seppl2013 added a comment.EditedSep 30 2019, 12:13 PM
cat /Users/wf/.pywikibot/families/royalfamily_family.py
# -*- coding: utf-8  -*-
from pywikibot import family

class Family(family.Family):
    name = 'royalfamily'
    langs = {
        'en': 'royal-family.bitplan.com',
    }
    def scriptpath(self, code):
        return ''

works - so it's looks the example is outdated.

I have added the example code to https://www.mediawiki.org/wiki/Manual:Pywikibot/Use_on_third-party_wikis#Configuring_custom_families_folder and recommend to close this issue.

Excellent! Thank you :)

so it's looks the example is outdated.

Yes, it seems so, we get rid of init() in Family classes recently. We should update docs about this issue as well

Dvorapa closed this task as Resolved.Sep 30 2019, 12:47 PM
Dvorapa claimed this task.
Dvorapa triaged this task as Medium priority.

so it's looks the example is outdated.

Yes, it seems so, we get rid of init() in Family classes recently. We should update docs about this issue as well

Done! Marking as resolved as we both updated docs to fix the issues found.

If you'll have any other questions, feel free to ask here or create a new task.