Page MenuHomePhabricator

Logging (pywiki module) always verbose if enabled
Closed, ResolvedPublicFeature

Assigned To
Authored By
Inductiveload
Jan 14 2021, 8:58 PM
Referenced Files
F35103737: bild.png
May 4 2022, 10:10 PM
Tokens
"Like" token, awarded by So9q."The World Burns" token, awarded by AntiCompositeNumber."Like" token, awarded by CONFIQ.

Description

I find I cannot change the pywiki module logging level, so it's always set to "Verbose" if I see and log level.

For example:

log_level = logging.INFO
logging.basicConfig(level=log_level)

# this doesn't work, I still get VERBOSE logging
logging.getLogger("pywiki").setLevel(logging.WARNING)

# this does, but I get no logging
pwbl = logging.getLogger("pywiki")
pwbl.disabled = True

Is there a way to set the same logging level as the surrounding code?

Event Timeline

Logging is a facility Pyhton module and part of ist standard library. How is this related to pywikibot framework and what dou you expect on our side? Probably use Python's bug Tracker: https://docs.python.org/3/bugs.html

@Xqt sure, but the following program should not produce any logging:

#! /usr/bin/env python3

import pywikibot
import logging

logging.basicConfig(level=logging.WARNING)

But instead it produces 298 lines that all start with VERBOSE:pywiki:

This is with Pywikibot 5.5.0

I see. 'pywiki' logger will be initialized to level 11 and there is no easy way to change that. As a work-around you can modify the logger after it was initialized e.g.

import logging
import pywikibot
pywikibot.output('This will initialize the logger')
logger = logging.getLogger('pywiki')
logger.setLevel(logging.WARNING)
pywikibot.output('This message will not be logged anymore')
pywikibot.log('Also verbose lgs are hidden')
pywikibot.warning('Warnings are still logged')
pywikibot.error('Errors are logged too.)
Xqt triaged this task as Low priority.Jan 15 2021, 2:16 PM
Xqt changed the subtype of this task from "Task" to "Feature Request".
Xqt raised the priority of this task from Low to High.Oct 16 2021, 8:44 AM

'pywiki' logger will be initialized to level 11 and there is no easy way to change that

Python's logging module offers a lot of options for configuration; would it make sense to move some of this hardcoded configuration to a config file?

'pywiki' logger will be initialized to level 11 and there is no easy way to change that

Python's logging module offers a lot of options for configuration; would it make sense to move some of this hardcoded configuration to a config file?

user-config.py is able to hold such settings already e.g. with the sample given above:

logger = logging.getLogger('pywiki')
logger.setLevel(logging.WARNING)

but a config file should be preferred anyway.

I see. 'pywiki' logger will be initialized to level 11 and there is no easy way to change that. As a work-around you can modify the logger after it was initialized e.g.

import logging
import pywikibot
pywikibot.output('This will initialize the logger')
logger = logging.getLogger('pywiki')
logger.setLevel(logging.WARNING)
pywikibot.output('This message will not be logged anymore')
pywikibot.log('Also verbose lgs are hidden')
pywikibot.warning('Warnings are still logged')
pywikibot.error('Errors are logged too.)

This workaround did not work for me :/

I commented out the whole content of bot.py: writelogheader() and that did the trick!

I see. 'pywiki' logger will be initialized to level 11 and there is no easy way to change that. As a work-around you can modify the logger after it was initialized e.g.

import logging
import pywikibot
pywikibot.output('This will initialize the logger')
logger = logging.getLogger('pywiki')
logger.setLevel(logging.WARNING)
pywikibot.output('This message will not be logged anymore')
pywikibot.log('Also verbose lgs are hidden')
pywikibot.warning('Warnings are still logged')
pywikibot.error('Errors are logged too.)

This workaround did not work for me :/

I commented out the whole content of bot.py: writelogheader() and that did the trick!

@So9q, starting to work on this item now; by "did not work" do you mean you still see the ~300 lines of verbose log output? What platform are you running on?

@maynorc: Are you still working on it? Do you need any help?

I see. 'pywiki' logger will be initialized to level 11 and there is no easy way to change that. As a work-around you can modify the logger after it was initialized e.g.

import logging
import pywikibot
pywikibot.output('This will initialize the logger')
logger = logging.getLogger('pywiki')
logger.setLevel(logging.WARNING)
pywikibot.output('This message will not be logged anymore')
pywikibot.log('Also verbose lgs are hidden')
pywikibot.warning('Warnings are still logged')
pywikibot.error('Errors are logged too.)

This workaround did not work for me :/

I commented out the whole content of bot.py: writelogheader() and that did the trick!

@So9q, starting to work on this item now; by "did not work" do you mean you still see the ~300 lines of verbose log output? What platform are you running on?

Yes, PyCharm on W10. Version 7.2.0
See the source code here https://github.com/internetarchive/wcdimportbot/blob/92ee7310a10195ce8d202a1b829ddb9a2afd8578/src/__init__.py#L18

bild.png (500×1 px, 167 KB)

I'm leaning towards forking pywikibot and removing the offending lines in bot.py causing the verbose log of files.

Sorry for the delay, hoping to get to it next week.

@Xqt , it appears that this issue has now been fixed. I checked that removing the one line changed in your bug fix reproduces the error: https://phabricator.wikimedia.org/T281643

With this line added back, I don't see issues with logging anymore on Mac or Windows 10. Unfortunately this change isn't in the latest 7.2.0 release, but once the version number is upgraded @So9q you should be good to go.

Apologies for not getting to this earlier

Xqt closed this task as Resolved.EditedMay 10 2022, 4:52 AM

Great. Thank you for exploring this. Comes with 7.3 in about two weeks.

This is quite a horror show. I do not even use pywikibot just import it and it throws a ton of log stuff at me. Unfortunately this is marked as resolved without given the clear answer what version makes sure that the logging is not verbose by default.

7.3 seems to work thanks god ...

Thanks for the update. Then I can retire my fork and upgrade to 7.3 in wcdimportbot 😀