GrowthExperiments has evolved significantly from its first commit. Unfortunately, our documentation hasn't evolved with it. This task is to discuss some ideas about how to improve the situation for the GrowthExperiments extension for two audiences: product (what can this extension do, what parts of the MediaWiki UI does it touch), and engineering (how does it do those things, what are the dependencies with other systems).
First let's look at what we have:
- https://www.mediawiki.org/wiki/Growth - Has a link to initiatives and a deployment table showing what is deployed where
- https://www.mediawiki.org/wiki/Extension:GrowthExperiments - contains one sentence overviews of dependencies on other extensions, and one sentence links to the initiatives as they map to features offered by the module
- https://www.mediawiki.org/wiki/Extension:GrowthExperiments/developer_setup - code snippets for how to get setup with GrowthExperiments in a local developer environment
To me, the things that are missing are:
- a clear overview of available features, with links to explore the relevant code and a list of dependencies
- for each feature, an explanation of how it can be configured in different ways, with screenshots to illustrate different possibilities (e.g. how help panel looks in normal mode vs in guidance mode)
- for each feature, links to relevant sections of code, and dependencies on internal and external services defined
- overview of what instrumentation exists, and links to dashboards where the data can be viewed
This documentation could conceivably be added to one or more of the wiki pages above, but:
- Translation makes writing documentation pages on mediawiki.org cumbersome -- can be worked around by explicitly marking the pages as non-translatable, but then we can't use the above pages
- The Extension:GrowthExperiments/developer_setup should probably be reworked as a set of scripts and a documentation file in the repository.
- the main Growth page contains a lot of information about research, community consultation, etc and it's hard to tease out the elements I'd like to document from that
- wiki pages are decoupled from code, so there is no inherent link between merging a patch and updating documentation.
I was thinking that we could use a markdown based documentation framework that publishes to doc.wikimedia.org/GrowthExperiments. The idea would be to have a common reference point for product/design/engineering on what features we have and how they are supposed to look and function, while also providing deeper documentation and references for engineers on where to find things.
We could use a git commit template that encourages patch creators to check a box about whether documentation is added or needs to be created.