Page MenuHomePhabricator

generator.close() does not work if generator is a propery
Open, In Progress, HighPublic

Description

Description

generator wrapper
In Pywikibot bots, BaseBot.run() assumes that self.generator is a mutable attribute, and may wrap it into a generator if it is only an iterable:

if not isinstance(self.generator, Generator):
    self.generator = (item for item in self.generator)

If generator is implemented as a @property without a setter, this breaks when wrapping is required, because run() attempts to assign to it and fails with:

AttributeError: property has no setter

generator.close()
A property may return a new generator instance on each access. This leads to inconsistent behavior:
run() may operate on one instance. self.generator.close() may affect a different instance. As a result: close() does not reliably stop iteration.

Expected behavior
run() must be able to assign a wrapped generator when needed. self.generator.close() must act on the active iterator.

Suggestions

  • Document that generator must be a settable attribute holding a stable iterator instance, not a property.
  • Warn when a property is detected.
  • re-implement the stop method like in rPWBCe1077727c3b (3.0.20200111) T198801, deprecated in rPWBC8d9bae8486 (4.1) and removed with rPWBC402abb24 (7.0)
  • use class tools.collections.GeneratorWrapper
  • A generator function or method works as expected when assigned as follows:
def __init__(self):
    self.generator = self.generator()

def generator(self):
    yield

Usualy an external generator function is passed to the BaseBot via generator option like

gen_factory = pagegenerators.GeneratorFactory()
bot = MyBot(generator=gen_factory.getCombinedGenerator())
bot.run()

If an internal generator should be used, then

  • it should not be a property but a function or a method. It should be assigned to the generator variable. Maybe if the Bot should provide its own generator, it should have a different identifier.
  • if it is a property, the generator.close() method cannot be used. A (deprecation) warning should be thrown.

Event Timeline

I looked into this. Found 7 scripts using generator as @property. Two of them (nowcommons.py, parser_function_count.py) return an iterable rather than yielding, so they'd hit the AttributeError on wrapping. The rest use yield, so they pass the isinstance(Generator) check but close() would act on a fresh instance.

From your suggestions, I think a combination of warning when a property is detected and documenting the contract would be a good start. I'd be happy to work on a patch if you're open to it.

From your suggestions, I think a combination of warning when a property is detected and documenting the contract would be a good start. I'd be happy to work on a patch if you're open to it.

Sure. The Idea is to use inspect.getattr_static to find out the generator`s type, something like this within Bot's initializer:

gentype = getattr_static(self, 'generator', None)
if gentype is None:
    pass
elif type(gentype) == property:
    issue_deprecation_warning(...)
elif callable(gentype):  # or better inspect.isgeneratorfunction(cls_attr) ???
    self.generator = gentype(self)  # only if it is not already set by generator keyword parameter
else:
    raise TypeError(...)
Xqt changed the task status from Open to In Progress.May 31 2026, 5:24 PM
Xqt claimed this task.