Page MenuHomePhabricator

Improve documentation for GrowthExperiments
Open, Stalled, LowPublic

Description

GrowthExperiments has evolved significantly from its first commit. Unfortunately, our documentation hasn't evolved with it. This task is to discuss some ideas about how to improve the situation for the GrowthExperiments extension for two audiences: product (what can this extension do, what parts of the MediaWiki UI does it touch), and engineering (how does it do those things, what are the dependencies with other systems).

First let's look at what we have:

To me, the things that are missing are:

  • a clear overview of available features, with links to explore the relevant code and a list of dependencies
  • for each feature, an explanation of how it can be configured in different ways, with screenshots to illustrate different possibilities (e.g. how help panel looks in normal mode vs in guidance mode)
  • for each feature, links to relevant sections of code, and dependencies on internal and external services defined
  • overview of what instrumentation exists, and links to dashboards where the data can be viewed

This documentation could conceivably be added to one or more of the wiki pages above, but:

  • Translation makes writing documentation pages on mediawiki.org cumbersome -- can be worked around by explicitly marking the pages as non-translatable, but then we can't use the above pages
  • The Extension:GrowthExperiments/developer_setup should probably be reworked as a set of scripts and a documentation file in the repository.
  • the main Growth page contains a lot of information about research, community consultation, etc and it's hard to tease out the elements I'd like to document from that
  • wiki pages are decoupled from code, so there is no inherent link between merging a patch and updating documentation.

I was thinking that we could use a markdown based documentation framework that publishes to doc.wikimedia.org/GrowthExperiments. The idea would be to have a common reference point for product/design/engineering on what features we have and how they are supposed to look and function, while also providing deeper documentation and references for engineers on where to find things.

We could use a git commit template that encourages patch creators to check a box about whether documentation is added or needs to be created.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
kostajh added subscribers: Sgs, DMburugu, mewoph, Tgr.

Discussion item for the engineers; moving to Triaged. cc @mewoph, @Sgs, @Tgr, @DMburugu

See also T296443: Create GrowthExperiments newcomer tasks backend architecture diagram. Also T220147: Homepage: write help page for Special:Homepage, T243432: Coordinate the redaction of the help page about Suggested edits, T240232: Provide best practices concerning writing help pages on wikis although those are more community-facing.

Personally I am somewhat skeptical of moving documentation into the repository. I think there are significant benefits to using the wiki: easier to invite participation from others, multilingual, better navigation, discoverability and SEO. The one significant benefit it gives is that it is easier to review the code and documentation together, and easier to keep track of whether the documentation has been updated, which is very helpful but can be achieved in other ways (and those other ways might be necessary anyway given that certain types of documentation are expected to be on mw.org or wikitech). Some other teams use a Definition of Done to make sure documentation is reviewed at the end of every task (e.g. FR-tech, WMCS).

DMburugu triaged this task as Medium priority.Dec 20 2022, 9:11 AM
DMburugu moved this task from Triaged to Current Maintenance Focus on the Growth-Team board.
DMburugu lowered the priority of this task from Medium to Low.Jan 9 2023, 5:01 PM
kostajh changed the task status from Open to Stalled.Jan 31 2023, 8:39 PM

I volunteered to reword this task or split it into subtasks to make it actionable. Marking as "Stalled" until then.