Page MenuHomePhabricator

pywikibot.exceptions.NoUsername: ERROR
Closed, InvalidPublic

Description

When I run replace.py and any other script in core branch via cron for Urdu Wikipedia, it throws following error:

No handlers could be found for logger "pywiki"
Traceback (most recent call last):
  File "/data/project/servicegroup/core/pwb.py", line 264, in <module>
    if not main():
  File "/data/project/servicegroup/core/pwb.py", line 257, in main
    run_python_file(filename, [filename] + args, argvu, file_package)
  File "/data/project/servicegroup/core/pwb.py", line 120, in run_python_file
    main_mod.__dict__)
  File "/data/project/servicegroup/core/scripts/replace.py", line 1191, in <module>
    main()
  File "/data/project/servicegroup/core/scripts/replace.py", line 1181, in main
    site.login()
  File "/mnt/nfs/labstore-secondary-tools-project/servicegroup/core/pywikibot/site.py", line 2090, in login
    user=self._username[sysop])
  File "/mnt/nfs/labstore-secondary-tools-project/servicegroup/core/pywikibot/tools/__init__.py", line 1520, in wrapper
    return obj(*__args, **__kw)
  File "/mnt/nfs/labstore-secondary-tools-project/servicegroup/core/pywikibot/login.py", line 122, in __init__
    'wiki_code': self.site.code})
pywikibot.exceptions.NoUsername: ERROR: Username for test:test is undefined.
If you have an account for that site, please add a line to user-config.py:

usernames['test']['test'] = 'myUsername'
CRITICAL: Closing network session.

Event Timeline

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptFeb 26 2018, 12:32 PM
matej_suchanek changed the task status from Open to Stalled.Feb 26 2018, 8:02 PM
matej_suchanek added a subscriber: matej_suchanek.

What environment do you use? Toolforge? What value have family and mylang in user-config.py? What command line arguments does the script run with? Does it happen when you explicitly provide -lang:ur -family:wikipedia?

zhuyifei1999 closed this task as Invalid.Feb 26 2018, 8:39 PM
zhuyifei1999 added a subscriber: zhuyifei1999.

If you have an account for that site, please add a line to user-config.py:
usernames['test']['test'] = 'myUsername'

... is self-explanatory. Follow the installation instructions. Reopen if that does not work somehow.

And it's expecting the site as test:test. Did you set family and mylang in your user-config.py?

And it's expecting the site as test:test. Did you set family and mylang in your user-config.py?

Yes, My user config is all set and when we run any script in core branch manually it works well. This error happens when I run them via cron job.

What environment do you use? Toolforge? What value have family and mylang in user-config.py? What command line arguments does the script run with? Does it happen when you explicitly provide -lang:ur -family:wikipedia?

Yes, I use toolforge. Family is wikipedia and mylang 'urwiki'.
Yes, it happens when I provide -lang:ur -family:wikipedia, but only in cronjob. Otherwise it works well.

MuhammadShuaib reopened this task as Open.Feb 27 2018, 5:48 AM
MuhammadShuaib added a comment.EditedFeb 27 2018, 7:07 AM

After providing -lang:ur -family:wikipedia in cron, it creates error file with following text

pywikibot.exceptions.NoUsername: ERROR: Username 
for wikipedia:ur is undefined.                   
If you have an account for that site, please add 
a line to user-config.py:                        
                                                 
usernames['wikipedia']['ur'] = 'myUsername'

And my user-config.py has already this line which placed in core folder.
Note that I am running pywikibot for several years, so i am very familiar of these requirements.

Dalba added a subscriber: Dalba.Feb 27 2018, 7:15 AM

What is the output of python pwb.py version?

What is the command that your are using to invoke the script?

Make sure you don't have another user-config.py in your working directory. (that will override the one in .pywikibot directory.)

MuhammadShuaib added a comment.EditedFeb 27 2018, 9:13 AM

What is the output of python pwb.py version?

Output is as follows:

 Pywikibot: [https] r-pywikibot-core.git (4ba8d64,
 g9120, 2018/02/26, 21:37:31, ok)                
Release version: 3.0-dev                         
requests version: 2.2.1                          
  cacerts: /etc/ssl/certs/ca-certificates.crt    
    certificate test: ok                         
Python: 2.7.6 (default, Nov 23 2017, 15:49:48)   
[GCC 4.8.4]                                      
Toolforge hostname: tools-bastion-03             
PYWIKIBOT2_DIR: Not set                          
PYWIKIBOT2_DIR_PWB:                              
PYWIKIBOT2_NO_USER_CONFIG: Not set               
Config base dir: /mnt/nfs/labstore-secondary-tool
s-project/servicegroup/core

What is the command that your are using to invoke the script?

python pwb.py replace -fix:correct-ur -transcludes:Infobox_settlement -lang:ur -family:wikipedia -always

Make sure you don't have another user-config.py in your working directory. (that will override the one in .pywikibot directory.)

Yes, I am sure.

Dalba added a comment.Feb 27 2018, 9:21 AM

Could you also give the result of cat user-config.py?

Could you also give the result of cat user-config.py?

Sure :)

# -*- coding: utf-8  -*-
console_encoding = 'utf-8'
textfile_encoding = 'unicode_escape'
family = 'wikipedia'
mylang = 'ur'
password_file= 'passfile'
usernames['wikipedia']['ur'] = u'KhanaMalumatBot'
transliteration_target = None
console_encoding = 'utf-8'
db_username = "*****"
db_password = "*****"
Dalba added a comment.EditedFeb 27 2018, 12:03 PM

Thanks! It seems to me that pywikibot is not able to find your user-config.py.

What is the path of your user-config.py ? (it normally should be the same directory as pwb.py or in ~/.pywikibot)

You could try setting the PYWIKIBOT2_DIR environment variable (export PYWIKIBOT2_DIR=/absolute/path/to/your/user-config/directory) and see if it helps (you should do this in your cron script).

Has this problem occurred recently or is it the first time you're running this script through the cron job?

Thanks! It seems to me that pywikibot is not able to find your user-config.py.
What is the path of your user-config.py ? (it normally should be the same directory as pwb.py or in ~/.pywikibot)

Yes, user-config.py and pwb.py are already in the same directory.

You could try setting the PYWIKIBOT2_DIR environment variable (export PYWIKIBOT2_DIR=/absolute/path/to/your/user-config/directory) and see if it helps (you should do this in your cron script).

I am running script via jsub command, how to do this in cron?

Has this problem occurred recently or is it the first time you're running this script through the cron job?

Recently I switched from compat to core branch and faced this problem.

Dalba added a comment.EditedFeb 27 2018, 2:42 PM

I am running script via jsub command, how to do this in cron?

One way to do it is to create a myscript.sh file containing:

#!/usr/bin/env bash
export PYWIKIBOT2_DIR=/absolute/path/to/your/user-config/directory
cd /path/to/your/pwb/directory
python pwb.py replace -fix:correct-ur -transcludes:Infobox_settlement -lang:ur -family:wikipedia -always

make it executable using chmod; and then add a line like the following to your crontab:

X Y * * * jsub ~/myscript.sh

python pwb.py replace -fix:correct-ur -transcludes:Infobox_settlement -lang:ur -family:wikipedia -always

I assume this is tool khanamalumat.

Yes, I am sure.

I see three user-config.py-s:

04:54:52 0 ✓ zhuyifei1999@tools-bastion-02: ~$ ls -l /data/project/khanamalumat/{,compat/,core/}user-config.py
-rw-r--r-- 1 tools.khanamalumat tools.khanamalumat 1217 Feb 26 06:25 /data/project/khanamalumat/compat/user-config.py
-rw-r--r-- 1 tools.khanamalumat tools.khanamalumat 1217 Feb 26 06:20 /data/project/khanamalumat/core/user-config.py
-rw-r--r-- 1 nadwi              tools.khanamalumat 1217 Feb 26 05:23 /data/project/khanamalumat/user-config.py

Note that when jsub executes a script it sets your current directory to your home directory unless otherwise specified. However, all these files are identical, so something weird is going on. (Also please see T188323)

This issue is indeed reproduce-able:

04:52:03 0 ✓ zhuyifei1999@tools-bastion-02: /data/project/khanamalumat$ python -i core/pwb.py 
No handlers could be found for logger "pywiki"
Wrapper script to use Pywikibot in 'directory' mode.

Run scripts using:

    python pwb.py <name_of_script> <options>

and it will use the package directory to store all user files, will fix up
search paths so the package does not need to be installed, etc.

>>> import pywikibot
>>> pywikibot.config.family
u'test'
>>>

Investigating

I'm pretty sure you hit config2.py#L1023, as the user-config.py in your home directory is not owned by the tool account.

05:11:11 0 ✓ zhuyifei1999@tools-bastion-02: /data/project/khanamalumat$ stat user-config.py 
  File: ‘user-config.py’
  Size: 1217      	Blocks: 8          IO Block: 1048576 regular file
Device: 26h/38d	Inode: 210895958   Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 3642/   nadwi)   Gid: (53670/tools.khanamalumat)
Access: 2018-02-26 05:28:52.000000000 +0000
Modify: 2018-02-26 05:23:57.000000000 +0000
Change: 2018-02-26 05:29:00.146963126 +0000
 Birth: -

Have you ran take, or better, de-duplicate your user-config.py-s?

What I'm still trying to figure out is why the warning was not shown.

Yep, if I pre-configure logging the warning is visible:

05:52:01 0 ✓ zhuyifei1999@tools-bastion-02: /data/project/khanamalumat$ python
Python 2.7.6 (default, Nov 23 2017, 15:49:48) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> 
>>> import sys
>>> sys.path.append('/mnt/nfs/labstore-secondary-tools-project/khanamalumat/core')
>>> 
>>> execfile('core/pwb.py', {'__file__':'core/pwb.py'})
WARNING:pywiki:Skipped '/mnt/nfs/labstore-secondary-tools-project/khanamalumat/user-config.py': owned by someone else.
WARNING:pywiki:family and mylang are not set.
Defaulting to family='test' and mylang='test'.
>>>
zhuyifei1999 closed this task as Invalid.Feb 27 2018, 6:08 PM

Follow up: T188417 on the root cause. Closing as invalid as pywikibot was expected to fail (explicit code), only that the warning was not shown, unexpectedly.

I am running script via jsub command, how to do this in cron?

One way to do it is to create a myscript.sh file containing:

#!/usr/bin/env bash
export PYWIKIBOT2_DIR=/absolute/path/to/your/user-config/directory
cd /path/to/your/pwb/directory
python pwb.py replace -fix:correct-ur -transcludes:Infobox_settlement -lang:ur -family:wikipedia -always

make it executable using chmod; and then add a line like the following to your crontab:

X Y * * * jsub ~/myscript.sh

Thanks for this help, I will try it out.

I'm pretty sure you hit config2.py#L1023, as the user-config.py in your home directory is not owned by the tool account.

05:11:11 0 ✓ zhuyifei1999@tools-bastion-02: /data/project/khanamalumat$ stat user-config.py 
  File: ‘user-config.py’
  Size: 1217      	Blocks: 8          IO Block: 1048576 regular file
Device: 26h/38d	Inode: 210895958   Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 3642/   nadwi)   Gid: (53670/tools.khanamalumat)
Access: 2018-02-26 05:28:52.000000000 +0000
Modify: 2018-02-26 05:23:57.000000000 +0000
Change: 2018-02-26 05:29:00.146963126 +0000
 Birth: -

Have you ran take, or better, de-duplicate your user-config.py-s?
What I'm still trying to figure out is why the warning was not shown.

I ran take now and removed one of three user-configs which is in data/project/khanamalumat.

python pwb.py replace -fix:correct-ur -transcludes:Infobox_settlement -lang:ur -family:wikipedia -always

I assume this is tool khanamalumat.

Yes it us khanamalumat, but I am facing the same problem in my another tools.shuaib-bot in core branch.

Yes, I am sure.

I see three user-config.py-s:

04:54:52 0 ✓ zhuyifei1999@tools-bastion-02: ~$ ls -l /data/project/khanamalumat/{,compat/,core/}user-config.py
-rw-r--r-- 1 tools.khanamalumat tools.khanamalumat 1217 Feb 26 06:25 /data/project/khanamalumat/compat/user-config.py
-rw-r--r-- 1 tools.khanamalumat tools.khanamalumat 1217 Feb 26 06:20 /data/project/khanamalumat/core/user-config.py
-rw-r--r-- 1 nadwi              tools.khanamalumat 1217 Feb 26 05:23 /data/project/khanamalumat/user-config.py

Note that when jsub executes a script it sets your current directory to your home directory unless otherwise specified. However, all these files are identical, so something weird is going on. (Also please see T188323)
This issue is indeed reproduce-able:

04:52:03 0 ✓ zhuyifei1999@tools-bastion-02: /data/project/khanamalumat$ python -i core/pwb.py 
No handlers could be found for logger "pywiki"
Wrapper script to use Pywikibot in 'directory' mode.
Run scripts using:
    python pwb.py <name_of_script> <options>
and it will use the package directory to store all user files, will fix up
search paths so the package does not need to be installed, etc.
>>> import pywikibot
>>> pywikibot.config.family
u'test'
>>>

Investigating

Yes, there are three, one is for compat and second for core and now I removed the third one.