Trying to get pywikibot running by following documentation fails
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Seppl2013
	Sep 28 2019, 6:07 PM

Description

user-config.py

# -*- coding: utf-8 -*-
import os
family = 'royalfamily'
register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family)
mylang = 'en'
usernames[family]['en'] = u'wf'
console_encoding = 'utf-8'
password_file = "wf-password.py"

migrate.py

#!/usr/bin/python
# -*- coding: utf-8 -*-
import pywikibot
mysite = pywikibot.Site()
print(mysite.sitename)

python3 migrate.py 
Traceback (most recent call last):
  File "migrate.py", line 4, in <module>
    mysite = pywikibot.Site()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/__init__.py", line 1237, in Site
    fam = Family.load(fam)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/tools/__init__.py", line 1738, in wrapper
    return obj(*__args, **__kw)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/family.py", line 1008, in load
    raise UnknownFamily('Family %s does not exist' % fam)
pywikibot.exceptions.UnknownFamily: Family royalfamily does not exist
CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.UnknownFamily'>

alternatively try

family_files[family] = 'http://royal-family.bitplan.com/'

to get

WARNING: /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/site.py:774: UserWarning: Global configuration variable "mylang" changed to "royalfamily" while instantiating site royalfamily:royalfamily
  % (self.__code, self), UserWarning)

WARNING: migrate.py:4: UserWarning: Site royalfamily:royalfamily instantiated using different code "en"
  mysite = pywikibot.Site()

royalfamily:royalfamily

none of this makes sense to me and i commented on the other phabricator tasks with similar issues that the documentation and the experience seems to be inconsistent.

Related Objects

Mentioned In: T245588: Third-party families cannot be registered in user-config.py
T130570: -user fails when -lang not set for single site family
Mentioned Here: T153891: generate_family_file.py crashes for a private wiki

Event Timeline

Seppl2013 created this task.Sep 28 2019, 6:07 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 28 2019, 6:07 PM

Seppl2013 updated the task description. (Show Details)Sep 28 2019, 6:08 PM

Seppl2013 updated the task description. (Show Details)

JJMC89 added a project: Pywikibot.Sep 28 2019, 6:09 PM

Restricted Application added a subscriber: pywikibot-bugs-list. · View Herald TranscriptSep 28 2019, 6:09 PM

Hi, please always explain which specific documentation you are following. Do you refer to https://www.mediawiki.org/wiki/Manual:Pywikibot/Use_on_third-party_wikis ?

alternatively try
family_files[family] = 'http://royal-family.bitplan.com/'

Try where? In user-config.py? Please be specific, thanks. :)

to get

There are some warnings, but you do not explain what the actual problem is. Please see and follow https://www.mediawiki.org/wiki/How_to_report_a_bug

@Seppl2013: Probably you should use generate_family_files.py to create you private family file.

thx for the comments. The actual problem is that the warnings do not make sense. They refer to something independent like Global configuration variable "mylang" and are therefor confusing for newbies. Yes https://www.mediawiki.org/wiki/Manual:Pywikibot/Use_on_third-party_wikis is the documentation I tried for this. Yes the alternative statement is in the same user-config.py file and commented out/in for the two attempts i made. I do not know how to use generate_family_files.py. The documentation says:

The generate_family_file.py in the root directory

Obviously it's not referrring to the root directory of my MacOS installation. But it looks like i am expected to search for this root directory. So a google search shows me https://phabricator.wikimedia.org/T153891

So I am expecting it will crash if i try it ... since that task is open since December 2016 ...

Ok I am continuing my search for the root directory in which to find generate_family.py - at https://www.mediawiki.org/wiki/Manual:Pywikibot/Scripts it's a red link ...
I do find it at https://phabricator.wikimedia.org/diffusion/PWBC/repository/master/ but i didn't install things from phabricator. I used a
requirements.txt

# 2019-09-28
Pywikibot

and did a

$pip install -r requirements.txt

making sure the right pip is used in this case the one for python3 ...

pip show Pywikibot

tells me a long path in wich I try to find the generate_family_file.py but I get an empty result on:

find /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages -name "generate_fam*"

It means the root directory of your Pywikibot installation

If you used pip to install Pywikibot, then this file generate_family_file.py is not included in your Pywikibot installation and you have to create your family file by hand in the families directory

I tried:

register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+".py")

in
user-config.py

# -*- coding: utf-8 -*-
import os
family = 'royalfamily'
register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+".py")
mylang = 'en'
usernames[family]['en'] = u'wf'
console_encoding = 'utf-8'
password_file = "wf-password.py"
#family_files[family] = 'http://royal-family.bitplan.com/'
#register_families_folder())
#use_api_login = True

and

cat royalfamily_family.py 
from pywikibot import family

class Family(family.Family):
    name = 'royalfamily'
    langs = {
        'en': 'http://royal-family.bitplan.com/'
    }

with the result:

python3 migrate.py 
Traceback (most recent call last):
  File "migrate.py", line 4, in <module>
    mysite = pywikibot.Site()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/__init__.py", line 1237, in Site
    fam = Family.load(fam)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/tools/__init__.py", line 1738, in wrapper
    return obj(*__args, **__kw)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/family.py", line 1008, in load
    raise UnknownFamily('Family %s does not exist' % fam)
pywikibot.exceptions.UnknownFamily: Family royalfamily does not exist
CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.UnknownFamily'>

Xqt changed the task status from Stalled to Open.Sep 30 2019, 10:25 AM

Issue is in your family file detection, because os.path.expanduser("~")+'/.pywikibot/families'+"/"+family+".py" gives royalfamily.py, not royalfamily_family.py

I assume your folder structure is like this (in your user folder, often abbreviated as ~):

.pywikibot
-- user_config.py
-- families
   -- royalfamily_family.py

You should use os.path.expanduser('~') + '/.pywikibot/families/' + family + '_family.py' instead. See:

$ python
Python 3.7.4 (default, Jul 16 2019, 07:12:58) 
[GCC 9.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import pywikibot  // no need to import in user_config.py
>>> from pywikibot import config2  // no need to import in user_config.py
>>> family = 'royalfamily'
>>> mylang = 'en'
>>> config2.register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+".py")
>>> pywikibot.Site(mylang, family)
Traceback (most recent call last):
  File "/home/pavel/.local/lib/python3.7/site-packages/pywikibot/family.py", line 1017, in load
    mod = import_module(splitext(basename(family_file))[0])
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'royalfamily'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pavel/.local/lib/python3.7/site-packages/pywikibot/__init__.py", line 1237, in Site
    fam = Family.load(fam)
  File "/home/pavel/.local/lib/python3.7/site-packages/pywikibot/tools/__init__.py", line 1738, in wrapper
    return obj(*__args, **__kw)
  File "/home/pavel/.local/lib/python3.7/site-packages/pywikibot/family.py", line 1019, in load
    raise UnknownFamily('Family %s does not exist' % fam)
pywikibot.exceptions.UnknownFamily: Family royalfamily does not exist
>>> config2.register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+"_family.py")
>>> pywikibot.Site(mylang, family)
APISite("en", "royalfamily")

Was it helpful? Have you succeeded? Can we close this?

Indeed I get the same "no module named royalfamily" error as you do with:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import os
import pywikibot  # no need to import in user_config.py
from pywikibot import config2  # no need to import in user_config.py
family = 'royalfamily'
mylang = 'en'
config2.register_family_file(family, os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+".py")
mysite=pywikibot.Site(mylang, family)
print(mysite.sitename)

So I improved it to:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import os
import pywikibot  # no need to import in user_config.py
from pywikibot import config2  # no need to import in user_config.py
family = 'royalfamily'
mylang = 'en'
familyfile=os.path.abspath(os.path.expanduser("~")+'/.pywikibot/families')+"/"+family+"_family.py"
if not os.path.isfile(familyfile):
  print ("family file %s is missing" % (familyfile))
else:
  print ("will try to use family file %s" % (familyfile))
config2.register_family_file(family, familyfile)
mysite=pywikibot.Site(mylang, family)
print(mysite.sitename)

With the expected result

python3 migrate.py 
will try to use family file /Users/wf/.pywikibot/families/royalfamily_family.py
royalfamily:en

I have added the example code to https://www.mediawiki.org/wiki/Manual:Pywikibot/Use_on_third-party_wikis#Configuring_custom_families_folder and recommend to close this issue.

Thanks a lot for the swift reactions and in depth discussion.

Sorry - the next issue is coming up:

WARNING: /opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pywikibot/family.py:71: _NotImplementedWarning: royalfamily_family.new__.init__ is deprecated. cls.instance.init()

Which looks like the syntax:

# -*- coding: utf-8  -*-
from pywikibot import family

class Family(family.Family):
    def __init__(self):
        self.name = 'royalfamily'
        self.langs = {
            'en': 'http://royal-family.bitplan.com/'
        }
        
    def scriptpath(self, code):
         return ''

is not o.k. although I just changed the doc page assuming the examples in https://www.mediawiki.org/wiki/Manual:Pywikibot/Third-party_Wiki_Quick_Start would be o.k.

cat /Users/wf/.pywikibot/families/royalfamily_family.py

# -*- coding: utf-8  -*-
from pywikibot import family

class Family(family.Family):
    name = 'royalfamily'
    langs = {
        'en': 'royal-family.bitplan.com',
    }
    def scriptpath(self, code):
        return ''

works - so it's looks the example is outdated.

I have added the example code to https://www.mediawiki.org/wiki/Manual:Pywikibot/Use_on_third-party_wikis#Configuring_custom_families_folder and recommend to close this issue.

Excellent! Thank you :)

so it's looks the example is outdated.

Yes, it seems so, we get rid of init() in Family classes recently. We should update docs about this issue as well

In T234135#5533876, @Dvorapa wrote:

so it's looks the example is outdated.

Yes, it seems so, we get rid of init() in Family classes recently. We should update docs about this issue as well

Done! Marking as resolved as we both updated docs to fix the issues found.

If you'll have any other questions, feel free to ask here or create a new task.

Seppl2013 mentioned this in T130570: -user fails when -lang not set for single site family.Sep 30 2019, 1:31 PM

desophos mentioned this in T245588: Third-party families cannot be registered in user-config.py.Feb 19 2020, 7:32 AM

Ernstkm subscribed.Nov 17 2020, 3:41 PM

Trying to get pywikibot running by following documentation failsClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Trying to get pywikibot running by following documentation fails
Closed, ResolvedPublic
Actions