Review incidents from the past year to determine how many had a testable regression as the cause. From review identify the top 15 target projects.
- [[ https://office.wikimedia.org/wiki/Release_Engineering/FY1819-Planning/Code_Health | Release Engineering/FY1819-Planning/Code Health ]] > FY1819-Q1 > From review identify the top 15 target projects.
- [[ https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/FY2019/TEC13:_Code_Health/Goals#Outcome_2_/_Output_2.2 | Wikimedia Technology/Annual Plans/FY2019/TEC13: Code Health/Goals > Outcome 2 / Output 2.2 ]]
- Review existing e2e test coverage.
- Define prioritization scheme.
- Prioritize e2e testing gaps.
# Review existing e2e test coverage
List of repositories with at least one Selenium test.
| **Repository** | **Language**
| mediawiki/core | JavaScript
| mediawiki/extensions/AdvancedSearch | JavaScript
| mediawiki/extensions/CentralAuth | Ruby
| mediawiki/extensions/CentralNotice | Ruby
| mediawiki/extensions/CirrusSearch | JavaScript
| mediawiki/extensions/Echo | JavaScript
| mediawiki/extensions/ElectronPdfService | JavaScript
| mediawiki/extensions/GettingStarted | Ruby
| mediawiki/extensions/Math | JavaScript
| mediawiki/extensions/MobileFrontend | Ruby
| mediawiki/extensions/MultimediaViewer | Ruby
| mediawiki/extensions/Newsletter | JavaScript
| mediawiki/extensions/ORES | JavaScript
| mediawiki/extensions/Popups | JavaScript
| mediawiki/extensions/QuickSurveys | Ruby
| mediawiki/extensions/RelatedArticles | JavaScript
| mediawiki/extensions/RevisionSlider | Ruby
| mediawiki/extensions/TwoColConflict | JavaScript, Ruby
| mediawiki/extensions/Wikibase | JavaScript, Ruby
| mediawiki/extensions/WikibaseLexeme | JavaScript, Ruby
| mediawiki/skins/MinervaNeue | Ruby
| phab-deployment | JavaScript
| wikimedia/portals/deploy | JavaScript
# Define prioritization scheme
- How much change did happen (in the last 12 months)?
- https://wikimedia.biterg.io
- Does a repository have maintainers?
- https://www.mediawiki.org/wiki/Developers/Maintainers
- {T190891}
- Do maintainers want tests?
- [[ https://docs.google.com/spreadsheets/d/1wcTLmgh8MzycmvjvgjY3ZGhxOYaXyDoY3NoNNv_v4Uw/edit#gid=0 | Incident Review 201709-201809 ]]
## How much change did happen
[[ https://wikimedia.biterg.io | Bitergia ]] report for the last year (P7586):
| Repository | Commits
| mediawiki/extensions | 35616
| operations/puppet | 7354
| mediawiki/core | 6288
| operations/mediawiki-config | 3160
| mediawiki/extensions/Wikibase | 2319
| mediawiki/skins | 2098
| apps/android/wikipedia | 1959
| phabricator/phabricator | 1532
| operations/software/librenms | 1446
| mediawiki/extensions/VisualEditor | 1263
| oojs/ui | 1229
| mediawiki/extensions/BlueSpiceExtensions | 1176
| integration/config | 1160
| pywikibot/core | 1101
| mediawiki/extensions/MobileFrontend | 923
| mediawiki/extensions/ContentTranslation | 915
| VisualEditor/VisualEditor | 856
| mediawiki/extensions/CirrusSearch | 688
| translatewiki | 674
| mediawiki/extensions/BlueSpiceFoundation | 622
| wikimedia/fundraising/crm | 621
| mediawiki/extensions/Popups | 603
| mediawiki/extensions/Flow | 563
| operations/dns | 539
| mediawiki/extensions/Translate | 510
| mediawiki/extensions/Echo | 504
| mediawiki/extensions/DonationInterface | 494
| mediawiki/services/parsoid | 444
| mediawiki/extensions/WikimediaMessages | 380
| mediawiki/extensions/CentralAuth | 376
| mediawiki/vagrant | 373
| mediawiki/extensions/AbuseFilter | 354
| mediawiki/extensions/Collection | 328
| mediawiki/services/restbase | 327
| mediawiki/extensions/UploadWizard | 311
| mediawiki/extensions/FlaggedRevs | 297
| mediawiki/extensions/TimedMediaHandler | 291
| mediawiki/extensions/ProofreadPage | 283
| mediawiki/extensions/UniversalLanguageSelector | 283
| mediawiki/extensions/Newsletter | 276
| mediawiki/skins/Vector | 275
| mediawiki/extensions/Cite | 271
| mediawiki/extensions/CodeMirror | 269
| integration/jenkins-job-builder | 252
| mediawiki/extensions/CentralNotice | 246
| mediawiki/extensions/Wikidata | 227
| mediawiki/extensions/ConfirmEdit | 225
| mediawiki/extensions/WikimediaEvents | 219
| mediawiki/extensions/Math | 215
| operations/debs/python-kafka | 214
# Incident Review 201709-201809
| Date | Service | Incident
| 2017-09-17 | mediawiki | 20170917-MediaWiki-API
| 2017-09-20 | elasticsearch | 20170920-Elasticsearch
| 2017-09-20 | logstash | 20170920-Logstash
| 2017-09-22 | mediawiki | 20170922-REST-API-alert-pageimage-update
| 2017-10-01 | labvirt | 20171001-labvirt1015
| 2017-10-17 | database | 20171017-webm-audio-mediatype
| 2017-10-18 | wqds | 20171018-wdqs
| 2017-11-16 | wikidata | 20171116-s5-dewiki-wikidata
| 2017-11-20 | ores | 20171120-Ext:ORES
| 2017-11-25 | statsd | 20171125-statsd
| 2017-11-28 | ores | 20171128-ORES
| 2017-11-30 | wdqs | 20171130-wdqs
| 2018-01-10 | swift | 20180110-swift
| 2018-01-11 | mediawiki | 20180111-LegacyEncoding
| 2018-01-29 | mediawiki | 20180129-MediaWiki
| 2018-02-06 | phabricator | 20180206-Phabricator
| 2018-02-14 | labvirt | 20180214-labvirt1008-failure
| 2018-02-20 | thumbor | 20180220-thumbor
| 2018-02-22 | wdqs | 20180222-wdqs
| 2018-02-26 | WikibaseQualityConstraints | 20180226-WikibaseQualityConstraints
| 2018-02-28 | puppetmaster | 20180228-puppetmaster
| 2018-02-29 | train | 20180229-Train-1.31.0-wmf.27
| 2018-03-02 | mediawiki | 20180302-Train
| 2018-03-06 | mediawiki | 20180306-MasterMergedInWMFBranch
| 2018-03-12 | varnish | 20180312-Cache-text
| 2018-03-14 | ores | 20180314-ORES
| 2018-04-04 | mathoid | 20180404-mathoid
| 2018-04-10 | database | 20180410-Deleting a page on enwiki
| 2018-04-10 | routing | 20180410-Routing
| 2018-04-23 | wqds | 20180423-wdqs
| 2018-05-22 | mediawiki | 20180522-MediaWiki
| 2018-05-24 | wikidata | 20180524-wikidata
| 2018-06-06 | mail | 20180606-mx
| 2018-06-15 | phabricator | 20180615-phabricator-vandalism
| 2018-06-25 | wqds | 20180625-wdqs
| 2018-06-26 | LoadBalancers | 20180626-LoadBalancers
| 2018-06-28 | dumps | 20180628-dumps
| 2018-07-04 | mathoid | 20180704-mathoid
| 2018-07-05 | EventLogging | 20180705-Eventlogging-in-Hive
| 2018-07-10 | mediawiki | 20180710-MediaWiki
| 2018-07-11 | mail | 20180711-fundraising undeliverable mail
| 2018-07-11 | kafka | 20180711-kafka-eqiad
| 2018-07-17 | train | 20180717-Train
| 2018-07-24 | train | 20180724-Train
| 2018-07-28 | eventlogging | 20180728-eventlogging
| 2018-07-31 | phabricator | 20180731-Phabricator
| 2018-08-08 | network | 20180808-Network
| 2018-08-21 | train | 20180821-Train
| 2018-09-18 | train | 20180918-train
{F26221063}
# Prioritize e2e testing gaps
| Repository | Commits | Maintainers | Selenium
| mediawiki/extensions | 35616 | N/A
| operations/puppet | 7354 | SRE
| mediawiki/core | 6288 | RelEng | JavaScript
| operations/mediawiki-config | 3160 | RelEng
| mediawiki/extensions/Wikibase | 2319 | WMDE Engineering | JavaScript, Ruby
| mediawiki/skins | 2098 | N/A
| apps/android/wikipedia | 1959 | Reading Infrastructure
| phabricator/phabricator | 1532 | RelEng
| operations/software/librenms | 1446 | SRE
| mediawiki/extensions/VisualEditor | 1263 | Contributors > Editing team
| oojs/ui | 1229 |
| mediawiki/extensions/BlueSpiceExtensions | 1176 |
| integration/config | 1160 | RelEng
| pywikibot/core | 1101 |
| mediawiki/extensions/MobileFrontend | 923 | Reading/Web/Team | Ruby
| mediawiki/extensions/ContentTranslation | 915 | Language engineering
| VisualEditor/VisualEditor | 856 | Contributors > Editing team
| mediawiki/extensions/CirrusSearch | 688 | Search Platform | JavaScript
| translatewiki | 674 | translatewiki
| mediawiki/extensions/BlueSpiceFoundation | 622 |
| wikimedia/fundraising/crm | 621 | Fundraising tech
| mediawiki/extensions/Popups | 603 | Reading | JavaScript
| mediawiki/extensions/Flow | 563 | Growth team
| operations/dns | 539 | SRE
| mediawiki/extensions/Translate | 510 | Language engineering
| mediawiki/extensions/Echo | 504 | Growth team | JavaScript
| mediawiki/extensions/DonationInterface | 494 | Fundraising tech
| mediawiki/services/parsoid | 444 | Contributors > Parsing team
| mediawiki/extensions/WikimediaMessages | 380 | Growth team
| mediawiki/extensions/CentralAuth | 376 | Unassigned | Ruby
| mediawiki/vagrant | 373 | RelEng
| mediawiki/extensions/AbuseFilter | 354 | MediaWiki Platform team
| mediawiki/extensions/Collection | 328 | Unassigned
| mediawiki/services/restbase | 327 | Wikimedia Services team
| mediawiki/extensions/UploadWizard | 311 | Reading > Multimedia team
| mediawiki/extensions/FlaggedRevs | 297 | Growth team (Pending)
| mediawiki/extensions/TimedMediaHandler | 291 | Readers Engineering
| mediawiki/extensions/ProofreadPage | 283 | Contributors > Editing team
| mediawiki/extensions/UniversalLanguageSelector | 283 | Language engineering
| mediawiki/extensions/Newsletter | 276 | Unassigned | JavaScript
| mediawiki/skins/Vector | 275 | Reading
| mediawiki/extensions/Cite | 271 | Contributors > Editing team
| mediawiki/extensions/CodeMirror | 269 | Community Tech
| integration/jenkins-job-builder | 252 | RelEng
| mediawiki/extensions/CentralNotice | 246 | Fundraising tech | Ruby
| mediawiki/extensions/Wikidata | 227 | WMDE Engineering
| mediawiki/extensions/ConfirmEdit | 225 | Contributors > Editing team
| mediawiki/extensions/WikimediaEvents | 219 | Unassigned
| mediawiki/extensions/Math | 215 | Contributors > Editing team | JavaScript
| operations/debs/python-kafka | 214 | SRE