Lead: apaskulin
Advising: Krinkle

Goals

• Evaluate whether an easy-to-use, container-free, quickstart workflow for setting up a local MediaWiki development environment makes it easier to get started contributing to MediaWiki. See the Project outcomes section.
• If the new quickstart workflow meets the success criteria, integrate it with the MediaWiki development environment docs.
• Collect information and feedback that can be used to improve all MediaWiki development environments. See the Project outcomes section.

Background

As of October 2023, the fastest way to set up a development environment for MediaWiki is to use Docker. However, for people who aren't familiar with Docker, using containers for a development environment adds complexity and can be a barrier to entry. For more background information, see T347347: Make "Quick" MW install a thing.

Priority

This project contributes to Wikimedia Foundation 2023-24 annual plan objective WE3: "Increase collaboration and efficiency among software developers by improving the development process for MediaWiki", specifically key result WE 3.2: "Increase by 20% the number of authors that have committed more than 5 patches across a specific set of MediaWiki repositories that are deployed to production." This project is tracked as hypothesis WE3.2.4.

Success criteria

1. Setup time
The new workflow should be quick to set up. We'll evaluate setup time by testing the workflow on Windows, Mac, and Linux.

2. Complexity
The new workflow should be less complex than a container-based workflow. While it's difficult to measure complexity, in this case we'll use the number of actions required to complete the setup. Since the number of steps can vary based on how the document groups them, using actions helps us focus on the concepts instead of the presentation.

3. Ease of use
The new workflow should be easy to set up and use. We'll measure this qualitatively by collecting feedback from contributors.

Scope

This project is focused on new contributors, less experienced contributors, and contributors with simple use cases. It does not address needs for development environments that reproduce Wikimedia production, require multiple wikis, or use uncommon extensions or skins.

Stakeholders

• WMF MediaWiki Engineering team
• WMF API Platform team
• WMF Technical Documentation team
• WMF MediaWiki Product
• MediaWiki developers (staff, affiliate, and volunteer)
• MediaWiki documentation contributors
• MediaWiki Stakeholders' Group

Project plan

• 1. Research and prep
  • Prepare a wiki page documenting the new workflow to use in testing
    • Draft
    • Review
    • Finalize
  • Document and test existing workflows
    • Decide on start and end states for success criteria
• 2. Test new workflow
  • with stakeholders
  • with new contributors
  • at the WMF MediaWiki Code Jam (December 2023)
• 3. Publish finding and recommendations. See the Project outcomes section.
  • Either deprecate or integrate the new workflow with the MediaWiki development environment docs
  • Evaluate the limitations of new and existing workflows. See the Project outcomes section.

Project outcomes

Project goal: Evaluate quickstart workflow

Evaluate whether an easy-to-use, container-free, quickstart workflow for setting up a local MediaWiki development environment makes it easier to get started contributing to MediaWiki compared to MediaWiki-Docker.

I measured ease of getting started through:

1. Setup time
2. Complexity
3. Ease of use (qualitative feedback)

1. Setup time

The local quickstart is approximately 30% faster to set up than MediaWiki-Docker, making the local quickstart the quickest documented way to set up a local development environment for MediaWiki.

Details

To measure setup time, I started from a state in which:

• git is installed
• a package manager is installed (such as Homebrew on Mac)
• available packages have been updated (such as via apt update)

The end state must have a local MediaWiki instance open in a browser with Vector enabled. These tests did not include setting up a developer account or SSH key.

Through testing on my WMF Mac laptop, I established a baseline for MediaWiki-Docker of 10 minutes. The MediaWiki-Docker docs give an estimated setup time of 15 minutes, but I decided to use the shorter time that I observed in my tests.

I tested the local quickstart using my WMF Mac laptop, a personal Windows laptop, and virtual Linux machines set up using Vagrant. I also conducted two user tests (one on Mac and one on Linux) with users who had never used a MediaWiki development environment before. The average results, rounded up to the nearest minute, were 7 minutes on Mac, 6 minutes on Linux, and 6 minutes on Windows.

2. Complexity

Compared with MediaWiki-Docker, the local quickstart is less complex, requiring approximately 40% fewer action to complete. The quickstart removes the need to edit files as part of the setup process and provides a single place to run commands that interact with MediaWiki, in contrast to the need to run some commands inside the MediaWiki container when using MediaWiki-Docker.

Details

It's difficult to measure complexity, so, in this case, I used the number of actions required to complete the workflow using the same start and end states as the setup-time measurement. Since the number of steps can vary based on how the document groups them, using actions helps us focus on the concepts instead of the presentation.

Based on the documentation, MediaWiki-Docker requires 9 actions:

1. Install Docker.
2. Clone MediaWiki.
3. Create .env file.
4. Add user ID and group ID to your .env file.
5. Start containers.
6. Install PHP dependencies.
7. Install MediaWiki. (On Windows, this is two actions.)
8. Clone Vector.
9. Enable Vector in LocalSettings.php

The local quickstart requires 5 actions:

1. Install PHP and Composer. (On Windows and Ubuntu 20, this is two actions.)
2. Clone MediaWiki and Vector.
3. Install PHP dependencies.
4. Install MediaWiki.
5. Start server.

3. Ease of use

The quickstart received positive feedback from 2 users in user tests, 2 users through comments on MediaWiki.org and Phabricator, and 6 users surveyed following the WMF MediaWiki CodeJam. The wiki page on MediaWiki.org has received 744 pageviews (ref) since first published in October 2023.

Details

Feedback highlights:

• "This is looking REALLY good. Very readable." - from MediaWiki.org
• "Too bad I didn't know this when I was getting started with MediaWiki." - from Phabricator

CodeJam survey:

Out of 54 participants, there were 16 responses to the survey. Of these 16 responders, only 3 reported setting up a new development environment; the other 13 used an existing development environment. Of the 3 responders who set up a new development environment (all of which submitted a patch as part of CodeJam), 2 chose the local quickstart, and 1 chose Docker/MediaWiki CLI. Of the 13 responders who used an existing environment, 4 used the quickstart, compared with 7 who used MediaWiki CLI or Docker, 1 who used Vagrant, and 1 who used custom scripts. All 6 responders who used the local quickstart reported being satisfied with the tooling and the documentation.

Project goal: Integrate the new docs

If the new quickstart workflow meets the success criteria, integrate it with the MediaWiki development environment docs.

I concluded that there was enough improvement in the success criteria to justify replacing the local installation section on How to become a MediaWiki Hacker with a link to the local quickstart.

Project goal: Collect information and feedback

Collect information and feedback that can be used to improve all MediaWiki development environments.

MediaWiki-Docker

Because MediaWiki-Docker is used in DEVELOPERS.md in the MediaWiki core repository, it is considered the default development environment for MediaWiki. The reach of DEVELOPERS.md is likely the reason why the MediaWiki-Docker wiki page has significantly more pageviews than other development environment pages or than the landing page that links to it.

Overall, I found MediaWiki-Docker quick and easy to set up using Docker Desktop on Mac. In the default configuration, MediaWiki-Docker offers similar functionality to the local quickstart, using SQLite and the standard process for installing extensions. These features work well for simple use cases, but don't support more complete needs like running multiple wikis. Further investigation into these complex use cases is needed to make recommendations about how to evolve current development environments to support them.

Short-term documentation improvements to the MediaWiki-Docker quickstart in DEVELOPERS.md:

1. Fix formatting errors and simplify language.
2. Improve the end state of the quickstart by including installing and enabling Vector, so that the quickstart process ends with a local MediaWiki instance you can interact with, instead of ending with the skin-not-found error page. This will reduce the number of actions in the MediaWiki-Docker workflow from 9 to 7.
3. Simplify step 3. Prepare .env file to two echo commands, one for Windows and one for Linux/Mac. If possible, move the Xdebug step to the Xdebug section. This will further reduce the number of actions in the MediaWiki-Docker workflow by one.
4. Add link to Local development quickstart as a container-free option.

Long-term documentation considerations:

1. Should the documentation in DEVELOPERS.md be moved to MediaWiki.org where it can be translated and more easily edited? This should be part of a decision that also considers the documentation in the /docs directory.
2. The extension-specific subpages of MediaWiki-Docker (such as MediaWiki-Docker/Extension/Echo) partially duplicate and are siloed from the primary extension documentation pages. How can this duplication be reduced and these two sets of pages more easily maintained together?

MediaWiki CLI

MediaWiki CLI's setup wizard provides a great user experience that could theoretically be expanded to offer many different types of development environments. However, maintaining a tool like MediaWiki CLI is more complex than a docker-compose file or a set of composer commands. Further investigation is required to assess the maintainability of MediaWiki CLI and evaluate whether it can become a replacement for MediaWiki-Docker.