We could use a MediaWiki sqlite database. I am guessing (hoping) that mediawiki has a test runner which sets up a test instance of MediaWiki to run tests against. If so, we should try to use the MediaWiki test framework to set up the MediaWiki instance on localhost. If not, wtf, someone needs to build one of those.

However, I dont see anything like that; their .travis.yml only appears to set up an empty database instance without a http interface. @hashar , am I wrong there? Or do you know if someone is working on building a way for the MediaWiki tests to run live tests against the http frontend processor? For pywikibot needs, it would be sufficient if it only connected api.php to a port; we dont need to access index.php.

At least for jenkins the setup of a mediawiki installation just for a testrun is contained in the following builders:

• mw-install-sqlite
• mw-apply-settings
• mw-run-update-script

So it is fairly straight forward to create a job that creates a local mediawiki install and then runs something else. Search for name: 'mwext-qunit' in integration/config.git/jjb/mediawiki-extensions.yaml for an actual example where it is used.

Triggered by T114421#1711600 I'll explain why this task is a good idea: See http://martinfowler.com/bliki/TestPyramid.html for something that is probably a good introduction. Automated tests that contact multiple hosts on the internet outside of your control ranks probably like this x: manual testing < x < automated browser / ui / full stack tests < integration tests < unit tests (there should be fewer tests to the left and a test should be done as far as possible to the right). Full stack tests for pywikibot were something like run pywikibot against a per testrun mediawiki installation like a bot user would against a live installation.