Review incidents from the past year to determine how many had a testable regression as the cause. From review identify the top 15 target projects.
- Release Engineering/FY1819-Planning/Code Health > FY1819-Q1 > From review identify the top 15 target projects.
- Wikimedia Technology/Annual Plans/FY2019/TEC13: Code Health/Goals > Outcome 2 / Output 2.2
- Review existing e2e test coverage.
- Define prioritization scheme.
- Prioritize e2e testing gaps.
Review existing e2e test coverage
List of repositories with at least one Selenium test.
| Repository | Language |
| mediawiki/core | JavaScript |
| mediawiki/extensions/AdvancedSearch | JavaScript |
| mediawiki/extensions/CentralAuth | Ruby |
| mediawiki/extensions/CentralNotice | Ruby |
| mediawiki/extensions/CirrusSearch | JavaScript |
| mediawiki/extensions/Echo | JavaScript |
| mediawiki/extensions/ElectronPdfService | JavaScript |
| mediawiki/extensions/GettingStarted | Ruby |
| mediawiki/extensions/Math | JavaScript |
| mediawiki/extensions/MobileFrontend | Ruby |
| mediawiki/extensions/MultimediaViewer | Ruby |
| mediawiki/extensions/Newsletter | JavaScript |
| mediawiki/extensions/ORES | JavaScript |
| mediawiki/extensions/Popups | JavaScript |
| mediawiki/extensions/QuickSurveys | Ruby |
| mediawiki/extensions/RelatedArticles | JavaScript |
| mediawiki/extensions/RevisionSlider | Ruby |
| mediawiki/extensions/TwoColConflict | JavaScript, Ruby |
| mediawiki/extensions/Wikibase | JavaScript, Ruby |
| mediawiki/extensions/WikibaseLexeme | JavaScript, Ruby |
| mediawiki/extensions/WikimediaEvents | PHP |
| mediawiki/skins/MinervaNeue | Ruby |
| phab-deployment | JavaScript |
| wikimedia/portals/deploy | JavaScript |
Define prioritization scheme
- How much change did happen (in the last 12 months)?
- https://wikimedia.biterg.io > Last 90 days > Absolute > From 2017-11-01 00:00:00.000 > To 2018-10-31 23:59:59.999 > Go
- Git > Overview > Repositories > raw: P7776
- View on Bitergia
- https://wikimedia.biterg.io > Last 90 days > Absolute > From 2017-11-01 00:00:00.000 > To 2018-10-31 23:59:59.999 > Go
- Does a repository have maintainers?
- https://www.mediawiki.org/wiki/Developers/Maintainers (See also T190891)
- Do maintainers want tests?
- Incident documentation
How much change did happen
wikimedia.biterg.io report from 2017-11-01 to 2018-10-31.
| Repository | Commits |
|---|---|
| mediawiki/extensions | 11300 |
| operations/puppet | 7988 |
| mediawiki/core | 4590 |
| operations/mediawiki-config | 4005 |
| integration/config | 1652 |
| operations/software/librenms | 1169 |
| pywikibot/core | 927 |
| mediawiki/extensions/Wikibase | 806 |
| apps/android/wikipedia | 789 |
| mediawiki/services/parsoid | 700 |
| mediawiki/extensions/VisualEditor | 692 |
| operations/dns | 653 |
| VisualEditor/VisualEditor | 599 |
| mediawiki/skins | 570 |
| mediawiki/extensions/MobileFrontend | 504 |
| mediawiki/extensions/ContentTranslation | 491 |
| translatewiki | 486 |
| oojs/ui | 469 |
| wikimedia/fundraising/crm | 457 |
| mediawiki/extensions/BlueSpiceFoundation | 414 |
| mediawiki/extensions/CirrusSearch | 357 |
| mediawiki/extensions/AbuseFilter | 306 |
| phabricator/phabricator | 302 |
| mediawiki/services/restbase | 290 |
| mediawiki/extensions/Flow | 232 |
| mediawiki/extensions/Echo | 223 |
| mediawiki/vagrant | 221 |
| mediawiki/extensions/Popups | 184 |
| mediawiki/extensions/Translate | 182 |
| mediawiki/extensions/DonationInterface | 180 |
| analytics/refinery | 178 |
| mediawiki/extensions/PageTriage | 177 |
| mediawiki/extensions/Cargo | 176 |
| mediawiki/tools/codesniffer | 156 |
| mediawiki/extensions/TimedMediaHandler | 152 |
| mediawiki/extensions/UniversalLanguageSelector | 142 |
| mediawiki/vendor | 140 |
| mediawiki/extensions/SocialProfile | 139 |
| analytics/refinery/source | 138 |
| operations/software | 137 |
| mediawiki/services/restbase/deploy | 136 |
| operations/debs/pybal | 123 |
| mediawiki/extensions/CentralAuth | 116 |
| mediawiki/tools/release | 116 |
| mediawiki/services/cxserver | 112 |
| mediawiki/extensions/BlueSpiceExtensions | 110 |
| mediawiki/extensions/WikimediaEvents | 110 |
| labs/private | 108 |
| operations/debs/python-kafka | 104 |
| labs/tools/heritage | 96 |
Incident Review 2017-11-01 to 2018-10-31
| Repository | Incidents | Maintainers |
|---|---|---|
| operations/puppet | 22 | SRE |
| mediawiki/core | 6 | Core Platform |
| operations/mediawiki-config | 4 | Release Engineering |
| mediawiki/extensions/Wikibase | 4 | WMDE |
| wikidata/query/rdf | 2 | WMDE |
| operations/debs/pybal | 2 | SRE |
| mediawiki/extensions/ORES | 2 | Growth |
| integration/config | 2 | Release Engineering |
| wikidata/query/blazegraph | 1 | WMDE |
| operations/software | 1 | SRE |
| operations/dns | 1 | SRE |
| mediawiki/vagrant | 1 | Release Engineering |
| mediawiki/tools/release | 1 | |
| mediawiki/services/ores/deploy | 1 | Growth |
| mediawiki/services/eventstreams | 1 | |
| mediawiki/extensions/WikibaseQualityConstraints | 1 | WMDE |
| mediawiki/extensions/PropertySuggester | 1 | WMDE |
| mediawiki/extensions/PageTriage | 1 | Growth |
| mediawiki/extensions/Cognate | 1 | WMDE |
| mediawiki/extensions/Babel | 1 | Editing |
| maps/tilerator/deploy | 1 | |
| maps/kartotherian/deploy | 1 | |
| integration/jenkins | 1 | Release Engineering |
| eventlogging | 1 | |
| analytics/refinery/source | 1 | Analytics |
| analytics/refinery | 1 | Analytics |
| All-Projects | 1 | |
Repository, Change, Maintainers, Coverage, Incidents, Selected
| Repository | Change | Maintainers | Coverage | Incidents | Selected |
|---|---|---|---|---|---|
| mediawiki/extensions | 11300 | ||||
| operations/puppet | 7988 | SRE | 22 | ||
| mediawiki/core | 4590 | Core Platform | JavaScript | 6 | |
| operations/mediawiki-config | 4005 | Release Engineering | 4 | ||
| integration/config | 1652 | Release Engineering | 2 | ||
| operations/software/librenms | 1169 | SRE | |||
| pywikibot/core | 927 | ||||
| mediawiki/extensions/Wikibase | 806 | WMDE | JavaScript, Ruby | 4 | |
| apps/android/wikipedia | 789 | ||||
| mediawiki/services/parsoid | 700 | Parsing | |||
| mediawiki/extensions/VisualEditor | 692 | Editing | โ | ||
| operations/dns | 653 | SRE | 1 | ||
| VisualEditor/VisualEditor | 599 | Editing | |||
| mediawiki/skins | 570 | Reading | |||
| mediawiki/extensions/MobileFrontend | 504 | Reading | Ruby | ||
| mediawiki/extensions/ContentTranslation | 491 | Language engineering | โ | ||
| translatewiki | 486 | ||||
| oojs/ui | 469 | ||||
| wikimedia/fundraising/crm | 457 | Fundraising tech | |||
| mediawiki/extensions/BlueSpiceFoundation | 414 | ||||
| mediawiki/extensions/CirrusSearch | 357 | Search Platform | JavaScript | ||
| mediawiki/extensions/AbuseFilter | 306 | Contributors | โ | ||
| phabricator/phabricator | 302 | Release Engineering | โ | ||
| mediawiki/services/restbase | 290 | Core Platform | |||
| mediawiki/extensions/Flow | 232 | Growth | โ | ||
| mediawiki/extensions/Echo | 223 | Growth | JavaScript | ||
| mediawiki/vagrant | 221 | Release Engineering | 1 | ||
| mediawiki/extensions/Popups | 184 | Reading | JavaScript | ||
| mediawiki/extensions/Translate | 182 | Language engineering | โ | ||
| mediawiki/extensions/DonationInterface | 180 | Fundraising tech | โ | ||
| analytics/refinery | 178 | Analytics | 1 | ||
| mediawiki/extensions/PageTriage | 177 | Growth | 1 | โ | |
| mediawiki/extensions/Cargo | 176 | ||||
| mediawiki/tools/codesniffer | 156 | ||||
| mediawiki/extensions/TimedMediaHandler | 152 | Reading | โ | ||
| mediawiki/extensions/UniversalLanguageSelector | 142 | Language engineering | โ | ||
| mediawiki/vendor | 140 | ||||
| mediawiki/extensions/SocialProfile | 139 | ||||
| analytics/refinery/source | 138 | Analytics | 1 | ||
| operations/software | 137 | SRE | 1 | ||
| mediawiki/services/restbase/deploy | 136 | Core Platform | |||
| operations/debs/pybal | 123 | SRE | 2 | ||
| mediawiki/extensions/CentralAuth | 116 | Ruby | |||
| mediawiki/tools/release | 116 | 1 | |||
| mediawiki/services/cxserver | 112 | ||||
| mediawiki/extensions/BlueSpiceExtensions | 110 | ||||
| mediawiki/extensions/WikimediaEvents | 110 | PHP | |||
| labs/private | 108 | ||||
| operations/debs/python-kafka | 104 | SRE | |||
| labs/tools/heritage | 96 | ||||
git rev-list --all --after="11/1/2017" --before="10/31/2018" --count
| Repository | Change | Maintainers | Coverage | Incidents | Selected |
|---|---|---|---|---|---|
| mediawiki/extensions/WikibaseQualityConstraints | 910 | WMDE | 1 | โ | |
| mediawiki/extensions/ORES | 364 | Growth | JavaScript | 2 | |
| wikidata/query/rdf | 204 | WMDE | 2 | ||
| mediawiki/extensions/Babel | 146 | Editing | 1 | โ | |
| mediawiki/services/ores/deploy | 84 | Growth | 1 | ||
| maps/kartotherian/deploy | 80 | 1 | |||
| mediawiki/extensions/PropertySuggester | 67 | WMDE | 1 | โ | |
| maps/tilerator/deploy | 61 | 1 | |||
| mediawiki/extensions/Cognate | 47 | WMDE | 1 | โ | |
| eventlogging | 26 | 1 | |||
| integration/jenkins | 19 | Release Engineering | 1 | ||
| mediawiki/services/eventstreams | 16 | 1 | |||
| wikidata/query/blazegraph | 10 | WMDE | 1 | ||
| All-Projects | 1 | ||||
Prioritize e2e testing gaps
| Repository | Change | Maintainers | Coverage | Incidents | Selected |
|---|---|---|---|---|---|
| mediawiki/extensions/VisualEditor | 692 | Editing | โ | ||
| mediawiki/extensions/ContentTranslation | 491 | Language engineering | โ | ||
| mediawiki/extensions/AbuseFilter | 306 | Contributors | โ | ||
| phabricator/phabricator | 302 | Release Engineering | โ | ||
| mediawiki/extensions/Flow | 232 | Growth | โ | ||
| mediawiki/extensions/Translate | 182 | Language engineering | โ | ||
| mediawiki/extensions/DonationInterface | 180 | Fundraising tech | โ | ||
| mediawiki/extensions/PageTriage | 177 | Growth | 1 | โ | |
| mediawiki/extensions/TimedMediaHandler | 152 | Reading | โ | ||
| mediawiki/extensions/UniversalLanguageSelector | 142 | Language engineering | โ | ||
| mediawiki/extensions/WikibaseQualityConstraints | 910 | WMDE | 1 | โ | |
| mediawiki/extensions/Babel | 146 | Editing | 1 | โ | |
| mediawiki/extensions/PropertySuggester | 67 | WMDE | 1 | โ | |
| mediawiki/extensions/Cognate | 47 | WMDE | 1 | โ |