Page MenuHomePhabricator

Create a "my first Pywikibot bot" tutorial for Toolforge
Open, MediumPublic


Wanted: A detailed but not technical jargon heavy guide to setting up a new Pywikibot tool.

[10:25]  <theglobetrotter>	How do I set up a bot on toolforge that edits the wiki at regular intervals?

This question was asked in the #wikimedia-cloud irc channel, unfortunately at a time when no one was providing answers. It is a good question though. There are many possible ways to answer, but a reasonably common solution would be to first create a pywikibot script to perform the edits and then to setup a cron job to run that script at some defined interval.

Tutorial should cover:

  • Standard tutorial header material (prerequisites, overview of steps)
  • Setting up Owner-only OAuth for pywikibot (maybe something to document on mw.o if not already there)
  • Using shared pywikibot code on Toolforge
  • Example of running a pywikibot provided script
  • Example of running a trivial custom script
  • Example of Grid Engine cron job
  • Example of Kubernetes CronJob object (might be a good excuse to work on some helper script for this)
  • Standard tutorial footer material
    • Choosing a license
    • Publishing source code
    • Adding a Tool:* documentation page
    • Where to look next for help with more advanced use cases

Should include:

  • Screenshots!
  • Cut-n-paste instructions for everything

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Steal some from Also I want us to support crons in PAWS, at which point it'll become much more useful...

and interactive bots can already be run there and are being run there

As an aside, I think we should be trying to enable the "my first Pywikibot bot" experience to be within PAWS. For that, there is , and WMID has created an Indonesian language booklet with pretty pictures, etc. We can merge the print quality version into the MediaWiki page.

When PAWS has cron functionality , it could even be the "my super-dooper critical Pywikibot bot" experience.

valhallasw lowered the priority of this task from Low to Lowest.May 27 2016, 12:47 PM
valhallasw moved this task from Triage to Backlog on the Toolforge board.
valhallasw raised the priority of this task from Lowest to Low.May 27 2016, 12:49 PM
Vaishnav980123 added a subscriber: Vaishnav980123.

I would like to work on this bug.

I would like to work on this bug.

@Vaishnav980123 awesome! Do you have experience with Tool Labs and Pywikibot already? I think it's totally ok if you don't, but you will need to seek out more help from others if not.

I think the main focus of the "My first X" tutorial series should be to help show people who are not familiar with Tool Labs detailed steps needed to make something useful. I think reading some tutorials from the Heroku getting started series might be good inspiration for the kind of thing I'm hoping we will end up with. The awesome part of doing things like this on wikitech is that we can get help from lots of people in making the directions more clear and informative. Don't worry about getting everything perfect on your first try. Just be bold, start a page on wikitech, and start working. It would probably be a good idea to make the page with the {{draft}} template at first so that people are warned if they find while looking for help.

@Xephyr826: Thanks! Please tell us if you have any specific questions or get stuck so we can provide input!

Thanks! I'll come back to this next week when I have more time.

This comment was removed by Xephyr826.

@bd808 or anyone, Two questions:

@bd808 or anyone, Two questions:

  • Should this tutorial use PAWS to introduce Pywikibot as suggested above?

Feel free to, though PAWS is quite different from Tool Labs in many aspects (eg. Grid), but users should be able to familiarize themselves with Pywikibot with PAWS.

Definitely not; that does not show how the tutorial is related to tool labs. Personally I'd recommend somewhere inside as it is where a few other "my first..." tutorials for tool labs live)

There is also a very old (and perhaps outdated) tutorial at I'd recommend rewriting that to wikitech and redirect this page to the new tutorial.

@Xephyr826: Thanks for working on this!
I see this got moved to , great!
Is there more work needed on this task? Or is this task "resolved"?

Is there more work needed on this task? Or is this task "resolved"?

I think there is quite a bit more work that can be done on this tutorial, but it is nice to have something for people to iterate on.

I'm back for another crack at this one :-)

Xephyr826 renamed this task from Create a "my first Pywikibot bot" tutorial for Tool Labs to Create a "my first Pywikibot bot" tutorial for Toolforge.Mar 24 2018, 8:41 PM
Xephyr826 removed a project: Cloud-Services.
^ I suggest using the shared pywikibot as dosumented at

User:Russell Blau/Using pywikibot on Labs Might be outdated.

It looks like written in the pmtpa times, but it's not too bad of a current guide for a shared install. From a quick glance only minor details like the bash prompt and the pywikibot branch are different.

mw:Manual:Pywikibot/Installation/Toolforge Detailed instructions but some parts marked outdated.

Documents the method of having a tool running its own copy of pywikibot.
Section Setup the webspace is hopelessly outdated (pmtpa instructions, when we still had a global apache).
Section Setup the job submission describes the conversion from toolserver command to toolforge command and is not relevant. Syntax like -m 512M is preferred over -l h_vmem=512M as the former also specifies virtual_free in addition to h_vmem. And note that 512M is now the default after T120517.
Section Automatic updating git on Toolforge is really unnecessary. If someone wants to auto-update, they should use the shared pywikibot, which is updated twice a day AFAIK.

FWIW, because of issues like, I do not personally use that export PYTHONPATH thing. It should be unnecessary anyways. If a custom environment must be specified, eg. for virtualenvs or C dependencies, I usually use workarounds, eg.

  • using /absolute/path/to/virtualenv/bin/python as the python interpretor
  • using .local as the virtualenv
  • using wrapper scripts like


  • compat isn't maintained anymore. I'd say don't even mention it... causes more confusion than good.
  • Section setup web-space & Setup job submission suffers from the same outdatedness as mw:Manual:Pywikibot/Installation/Toolforge.

Expanded on the instructions to request access to Toolforge:

Pulled in simplified instructions from a few places so all the instructions are in one section.

both pretty good

both interior to the other in some aspects.

  • Should use /shared/pywikibot/core (if grid-only) or /data/project/shared/pywikibot/core (works also in k8s, but I very few run bots in k8s...)
  • Modifying PYTHONPATH is evil. There is no guarantee that bash_profile will be loaded at all. (T134495#4078967, T164277: jsub/jstart inconsistency: non-continuous jobs spawns a login bash shell that loads .bash_profile, but continuous jobs doesn't load either .bash_profile or .bashrc)
    • Highly recommend either using venv and/or the in the shared nightly clone.
    • Honestly, some people tell you to modify env vars with .bashrc, some .bash_profile, some .profile, and some .bash_aliases. The outcome? A terrible mess. (My opinion: .bash_aliases should go into .bashrc and .bashrc should be loaded by .profile after testing if the shell is bash; .bash_profile should go into .profile. Keep login shell configuration in .profile and non-login shell configuration in .bashrc)
  • Doesn't explain T60784#3925224 (Granted, people are very unlikely to hit this bug if they use pywikibot's own scripts, but if some more advanced people start to code their own pywikibot scripts and start adding some print statements, oh, gotta expect some fun...)
  • is evil. (my opinion doesn't help does it? admittedly, the script does help newcomers a lot, so, ignore this complaint)
  • Should seriously stop talking about compat. It ain't supported.
  • Setup web-space is like 4 years outdated. I'd recommend just remove the section.
  • qcronsub?! Is this toolserver?
  • Why h_vmem when we have -mem?
  • 256MB was default limit... until 3 years ago
  • Should start talking about python3 rather than this aged python2

tl;dr: badly outdated.

  • Nice docs on how to access toolforge
  • Local clone is the only way? This should not be used unless you're developing pywikibot code itself.
  • The instruction for webpage... while it works, it has nothing to do with pywikibot.
  • Doesn't even talk about how to submit a job to wither grid or k8s.
  • And yeah... empty sections

tl;dr: nice start, but bad finish, lacking in so much stuffs that should be said.

Looking forward to it :)

@zhuyifei1999 Agree, python2, compat, 2.0, all these things should be eliminated by force from Pywikibot manuals (and brutal force to still occurring pywikipedia and trunk or rewrite! Are we still at the beginning of the century?)

("screenshots"? of command line interface? wtf?)

Retaking this task.

Will be looking at the current tutorial and for updates and improvements.

See staging doc here:

FYI, in case it helps prioritize this task, we just had a Bot development workshop with South Asia community members and was recommended and listed as the resource for them to consult.

Xqt raised the priority of this task from Low to Medium.Mar 3 2021, 5:42 PM

@srodlund: are you still working on it?

@Xqt to be honest, I started working on this a while ago when I was working on some pywikibot tutorials for PAWS, but it has been something I haven't made much progress on lately. I'm going to unassign myself from this. If you are interested in working on it, I'm happy to collaborate.

Here's the original page on Wikitech:
And here's the document I started to work on: