Change Details

{F26477269} Review incidents from the past year to determine how many had a testable regression as the cause. From review identify the top 15 target projects. - [[ https://office.wikimedia.org/wiki/Release_Engineering/FY1819-Planning/Code_Health | Release Engineering/FY1819-Planning/Code Health ]] > FY1819-Q1 > From review identify the top 15 target projects. - [[ https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/FY2019/TEC13:_Code_Health/Goals#Outcome_2_/_Output_2.2 | Wikimedia Technology/Annual Plans/FY2019/TEC13: Code Health/Goals > Outcome 2 / Output 2.2 ]] - Review existing e2e test coverage. - Define prioritization scheme. - Prioritize e2e testing gaps. # Review existing e2e test coverage List of repositories with at least one Selenium test. | **Repository** | **Language** | mediawiki/core | JavaScript | mediawiki/extensions/AdvancedSearch | JavaScript | mediawiki/extensions/CentralAuth | Ruby | mediawiki/extensions/CentralNotice | Ruby | mediawiki/extensions/CirrusSearch | JavaScript | mediawiki/extensions/Echo | JavaScript | mediawiki/extensions/ElectronPdfService | JavaScript | mediawiki/extensions/GettingStarted | Ruby | mediawiki/extensions/Math | JavaScript | mediawiki/extensions/MobileFrontend | Ruby | mediawiki/extensions/MultimediaViewer | Ruby | mediawiki/extensions/Newsletter | JavaScript | mediawiki/extensions/ORES | JavaScript | mediawiki/extensions/Popups | JavaScript | mediawiki/extensions/QuickSurveys | Ruby | mediawiki/extensions/RelatedArticles | JavaScript | mediawiki/extensions/RevisionSlider | Ruby | mediawiki/extensions/TwoColConflict | JavaScript, Ruby | mediawiki/extensions/Wikibase | JavaScript, Ruby | mediawiki/extensions/WikibaseLexeme | JavaScript, Ruby | mediawiki/extensions/WikimediaEvents | PHP | mediawiki/skins/MinervaNeue | Ruby | phab-deployment | JavaScript | wikimedia/portals/deploy | JavaScript # Define prioritization scheme - How much change did happen (in the last 12 months)? - https://wikimedia.biterg.io > Last 90 days > Absolute > From 2017-11-01 00:00:00.000 > To 2018-10-31 23:59:59.999 > Go - Git > Overview > Repositories > raw: P7776 - Git > Attraction/Retention > Top Repositories > raw: P7777 - Gerrit > Overview > Repositories > raw: P7778 - Does a repository have maintainers? - https://www.mediawiki.org/wiki/Developers/Maintainers - {T190891} - Do maintainers want tests? - [[ https://docs.google.com/spreadsheets/d/1l6zyuV-vbJM1P3qHAqaMfegpECeItJeKN9ToUdl8jmk/edit?usp=sharing | Incident Review 201711-201810 ]] ## How much change did happen [[ https://wikimedia.biterg.io | Bitergia ]] report for the last year (P7586): | Repository | Commits | mediawiki/extensions | 35616 | operations/puppet | 7354 | mediawiki/core | 6288 | operations/mediawiki-config | 3160 | mediawiki/extensions/Wikibase | 2319 | mediawiki/skins | 2098 | apps/android/wikipedia | 1959 | phabricator/phabricator | 1532 | operations/software/librenms | 1446 | mediawiki/extensions/VisualEditor | 1263 | oojs/ui | 1229 | mediawiki/extensions/BlueSpiceExtensions | 1176 | integration/config | 1160 | pywikibot/core | 1101 | mediawiki/extensions/MobileFrontend | 923 | mediawiki/extensions/ContentTranslation | 915 | VisualEditor/VisualEditor | 856 | mediawiki/extensions/CirrusSearch | 688 | translatewiki | 674 | mediawiki/extensions/BlueSpiceFoundation | 622 | wikimedia/fundraising/crm | 621 | mediawiki/extensions/Popups | 603 | mediawiki/extensions/Flow | 563 | operations/dns | 539 | mediawiki/extensions/Translate | 510 | mediawiki/extensions/Echo | 504 | mediawiki/extensions/DonationInterface | 494 | mediawiki/services/parsoid | 444 | mediawiki/extensions/WikimediaMessages | 380 | mediawiki/extensions/CentralAuth | 376 | mediawiki/vagrant | 373 | mediawiki/extensions/AbuseFilter | 354 | mediawiki/extensions/Collection | 328 | mediawiki/services/restbase | 327 | mediawiki/extensions/UploadWizard | 311 | mediawiki/extensions/FlaggedRevs | 297 | mediawiki/extensions/TimedMediaHandler | 291 | mediawiki/extensions/ProofreadPage | 283 | mediawiki/extensions/UniversalLanguageSelector | 283 | mediawiki/extensions/Newsletter | 276 | mediawiki/skins/Vector | 275 | mediawiki/extensions/Cite | 271 | mediawiki/extensions/CodeMirror | 269 | integration/jenkins-job-builder | 252 | mediawiki/extensions/CentralNotice | 246 | mediawiki/extensions/Wikidata | 227 | mediawiki/extensions/ConfirmEdit | 225 | mediawiki/extensions/WikimediaEvents | 219 | mediawiki/extensions/Math | 215 | operations/debs/python-kafka | 214 # Incident Review 201709-201809 | Date | Service | Incident | 2017-09-17 | mediawiki | 20170917-MediaWiki-API | 2017-09-20 | elasticsearch | 20170920-Elasticsearch | 2017-09-20 | logstash | 20170920-Logstash | 2017-09-22 | mediawiki | 20170922-REST-API-alert-pageimage-update | 2017-10-01 | labvirt | 20171001-labvirt1015 | 2017-10-17 | database | 20171017-webm-audio-mediatype | 2017-10-18 | wqds | 20171018-wdqs | 2017-11-16 | wikidata | 20171116-s5-dewiki-wikidata | 2017-11-20 | ores | 20171120-Ext:ORES | 2017-11-25 | statsd | 20171125-statsd | 2017-11-28 | ores | 20171128-ORES | 2017-11-30 | wdqs | 20171130-wdqs | 2018-01-10 | swift | 20180110-swift | 2018-01-11 | mediawiki | 20180111-LegacyEncoding | 2018-01-29 | mediawiki | 20180129-MediaWiki | 2018-02-06 | phabricator | 20180206-Phabricator | 2018-02-14 | labvirt | 20180214-labvirt1008-failure | 2018-02-20 | thumbor | 20180220-thumbor | 2018-02-22 | wdqs | 20180222-wdqs | 2018-02-26 | WikibaseQualityConstraints | 20180226-WikibaseQualityConstraints | 2018-02-28 | puppetmaster | 20180228-puppetmaster | 2018-02-29 | train | 20180229-Train-1.31.0-wmf.27 | 2018-03-02 | mediawiki | 20180302-Train | 2018-03-06 | mediawiki | 20180306-MasterMergedInWMFBranch | 2018-03-12 | varnish | 20180312-Cache-text | 2018-03-14 | ores | 20180314-ORES | 2018-04-04 | mathoid | 20180404-mathoid | 2018-04-10 | database | 20180410-Deleting a page on enwiki | 2018-04-10 | routing | 20180410-Routing | 2018-04-23 | wqds | 20180423-wdqs | 2018-05-22 | mediawiki | 20180522-MediaWiki | 2018-05-24 | wikidata | 20180524-wikidata | 2018-06-06 | mail | 20180606-mx | 2018-06-15 | phabricator | 20180615-phabricator-vandalism | 2018-06-25 | wqds | 20180625-wdqs | 2018-06-26 | LoadBalancers | 20180626-LoadBalancers | 2018-06-28 | dumps | 20180628-dumps | 2018-07-04 | mathoid | 20180704-mathoid | 2018-07-05 | EventLogging | 20180705-Eventlogging-in-Hive | 2018-07-10 | mediawiki | 20180710-MediaWiki | 2018-07-11 | mail | 20180711-fundraising undeliverable mail | 2018-07-11 | kafka | 20180711-kafka-eqiad | 2018-07-17 | train | 20180717-Train | 2018-07-24 | train | 20180724-Train | 2018-07-28 | eventlogging | 20180728-eventlogging | 2018-07-31 | phabricator | 20180731-Phabricator | 2018-08-08 | network | 20180808-Network | 2018-08-21 | train | 20180821-Train | 2018-09-18 | train | 20180918-train | Service | Incident | mediawiki | 8 | wqds | 5 | train | 5 | phabricator | 3 | ores | 3 | wikidata | 2 | mathoid | 2 | mail | 2 | labvirt | 2 | EventLogging | 2 | database | 2 | WikibaseQualityConstraints | 1 | varnish | 1 | thumbor | 1 | swift | 1 | statsd | 1 | routing | 1 | puppetmaster | 1 | network | 1 | logstash | 1 | LoadBalancers | 1 | kafka | 1 | elasticsearch | 1 | dumps | 1 {F26229059} # Prioritize e2e testing gaps | Repository | Change (commits) | Maintainers | Coverage (Selenium) | Incidents | Top 15 | mediawiki/extensions | 35616 | N/A | N/A | N/A | N/A | operations/puppet | 7354 | SRE | N/A | N/A | N/A | mediawiki/core | 6288 | RelEng | JavaScript | 8 | operations/mediawiki-config | 3160 | RelEng | N/A | N/A | N/A | mediawiki/extensions/Wikibase | 2319 | WMDE Engineering | JavaScript, Ruby | mediawiki/skins | 2098 | N/A | N/A | N/A | N/A | apps/android/wikipedia | 1959 | Reading Infrastructure | N/A | N/A | N/A | phabricator/phabricator | 1532 | RelEng | N/A | N/A | 3 | operations/software/librenms | 1446 | SRE | mediawiki/extensions/VisualEditor | 1263 | Contributors > Editing team | oojs/ui | 1229 | | mediawiki/extensions/BlueSpiceExtensions | 1176 | | integration/config | 1160 | RelEng | N/A | N/A | N/A | pywikibot/core | 1101 | | N/A | N/A | N/A | mediawiki/extensions/MobileFrontend | 923 | Reading/Web/Team | Ruby | mediawiki/extensions/ContentTranslation | 915 | Language engineering | VisualEditor/VisualEditor | 856 | Contributors > Editing team | mediawiki/extensions/CirrusSearch | 688 | Search Platform | JavaScript | 1 | translatewiki | 674 | translatewiki | N/A | N/A | N/A | mediawiki/extensions/BlueSpiceFoundation | 622 | | wikimedia/fundraising/crm | 621 | Fundraising tech | mediawiki/extensions/Popups | 603 | Reading | JavaScript | mediawiki/extensions/Flow | 563 | Growth team | operations/dns | 539 | SRE | N/A | N/A | N/A | mediawiki/extensions/Translate | 510 | Language engineering | mediawiki/extensions/Echo | 504 | Growth team | JavaScript | mediawiki/extensions/DonationInterface | 494 | Fundraising tech | mediawiki/services/parsoid | 444 | Contributors > Parsing team | mediawiki/extensions/WikimediaMessages | 380 | Growth team | mediawiki/extensions/CentralAuth | 376 | Unassigned | Ruby | mediawiki/vagrant | 373 | RelEng | N/A | N/A | N/A | mediawiki/extensions/AbuseFilter | 354 | MediaWiki Platform team | mediawiki/extensions/Collection | 328 | Unassigned | mediawiki/services/restbase | 327 | Wikimedia Services team | mediawiki/extensions/UploadWizard | 311 | Reading > Multimedia team | mediawiki/extensions/FlaggedRevs | 297 | Growth team (Pending) | mediawiki/extensions/TimedMediaHandler | 291 | Readers Engineering | mediawiki/extensions/ProofreadPage | 283 | Contributors > Editing team | mediawiki/extensions/UniversalLanguageSelector | 283 | Language engineering | mediawiki/extensions/Newsletter | 276 | Unassigned | JavaScript | mediawiki/skins/Vector | 275 | Reading | mediawiki/extensions/Cite | 271 | Contributors > Editing team | mediawiki/extensions/CodeMirror | 269 | Community Tech | integration/jenkins-job-builder | 252 | RelEng | N/A | N/A | N/A | mediawiki/extensions/CentralNotice | 246 | Fundraising tech | Ruby | mediawiki/extensions/Wikidata | 227 | WMDE Engineering | | 2 | mediawiki/extensions/ConfirmEdit | 225 | Contributors > Editing team | mediawiki/extensions/WikimediaEvents | 219 | Unassigned | PHP | mediawiki/extensions/Math | 215 | Contributors > Editing team | JavaScript | operations/debs/python-kafka | 214 | SRE | N/A | N/A | 1

{F26477269} Review incidents from the past year to determine how many had a testable regression as the cause. From review identify the top 15 target projects. - [[ https://office.wikimedia.org/wiki/Release_Engineering/FY1819-Planning/Code_Health | Release Engineering/FY1819-Planning/Code Health ]] > FY1819-Q1 > From review identify the top 15 target projects. - [[ https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/FY2019/TEC13:_Code_Health/Goals#Outcome_2_/_Output_2.2 | Wikimedia Technology/Annual Plans/FY2019/TEC13: Code Health/Goals > Outcome 2 / Output 2.2 ]] - Review existing e2e test coverage. - Define prioritization scheme. - Prioritize e2e testing gaps. # Review existing e2e test coverage List of repositories with at least one Selenium test. | **Repository** | **Language** | mediawiki/core | JavaScript | mediawiki/extensions/AdvancedSearch | JavaScript | mediawiki/extensions/CentralAuth | Ruby | mediawiki/extensions/CentralNotice | Ruby | mediawiki/extensions/CirrusSearch | JavaScript | mediawiki/extensions/Echo | JavaScript | mediawiki/extensions/ElectronPdfService | JavaScript | mediawiki/extensions/GettingStarted | Ruby | mediawiki/extensions/Math | JavaScript | mediawiki/extensions/MobileFrontend | Ruby | mediawiki/extensions/MultimediaViewer | Ruby | mediawiki/extensions/Newsletter | JavaScript | mediawiki/extensions/ORES | JavaScript | mediawiki/extensions/Popups | JavaScript | mediawiki/extensions/QuickSurveys | Ruby | mediawiki/extensions/RelatedArticles | JavaScript | mediawiki/extensions/RevisionSlider | Ruby | mediawiki/extensions/TwoColConflict | JavaScript, Ruby | mediawiki/extensions/Wikibase | JavaScript, Ruby | mediawiki/extensions/WikibaseLexeme | JavaScript, Ruby | mediawiki/extensions/WikimediaEvents | PHP | mediawiki/skins/MinervaNeue | Ruby | phab-deployment | JavaScript | wikimedia/portals/deploy | JavaScript # Define prioritization scheme - How much change did happen (in the last 12 months)? - https://wikimedia.biterg.io > Last 90 days > Absolute > From 2017-11-01 00:00:00.000 > To 2018-10-31 23:59:59.999 > Go - Git > Overview > Repositories > raw: P7776 - Git > Attraction/Retention > Top Repositories > raw: P7777 - Gerrit > Overview > Repositories > raw: P7778 - Does a repository have maintainers? - https://www.mediawiki.org/wiki/Developers/Maintainers (See also T190891) - Do maintainers want tests? - [[ https://docs.google.com/spreadsheets/d/1l6zyuV-vbJM1P3qHAqaMfegpECeItJeKN9ToUdl8jmk/edit?usp=sharing | Incident Review 201711-201810 ]] ## How much change did happen [[ https://wikimedia.biterg.io | Bitergia ]] report for the last year (P7586): | Repository | Commits | mediawiki/extensions | 35616 | operations/puppet | 7354 | mediawiki/core | 6288 | operations/mediawiki-config | 3160 | mediawiki/extensions/Wikibase | 2319 | mediawiki/skins | 2098 | apps/android/wikipedia | 1959 | phabricator/phabricator | 1532 | operations/software/librenms | 1446 | mediawiki/extensions/VisualEditor | 1263 | oojs/ui | 1229 | mediawiki/extensions/BlueSpiceExtensions | 1176 | integration/config | 1160 | pywikibot/core | 1101 | mediawiki/extensions/MobileFrontend | 923 | mediawiki/extensions/ContentTranslation | 915 | VisualEditor/VisualEditor | 856 | mediawiki/extensions/CirrusSearch | 688 | translatewiki | 674 | mediawiki/extensions/BlueSpiceFoundation | 622 | wikimedia/fundraising/crm | 621 | mediawiki/extensions/Popups | 603 | mediawiki/extensions/Flow | 563 | operations/dns | 539 | mediawiki/extensions/Translate | 510 | mediawiki/extensions/Echo | 504 | mediawiki/extensions/DonationInterface | 494 | mediawiki/services/parsoid | 444 | mediawiki/extensions/WikimediaMessages | 380 | mediawiki/extensions/CentralAuth | 376 | mediawiki/vagrant | 373 | mediawiki/extensions/AbuseFilter | 354 | mediawiki/extensions/Collection | 328 | mediawiki/services/restbase | 327 | mediawiki/extensions/UploadWizard | 311 | mediawiki/extensions/FlaggedRevs | 297 | mediawiki/extensions/TimedMediaHandler | 291 | mediawiki/extensions/ProofreadPage | 283 | mediawiki/extensions/UniversalLanguageSelector | 283 | mediawiki/extensions/Newsletter | 276 | mediawiki/skins/Vector | 275 | mediawiki/extensions/Cite | 271 | mediawiki/extensions/CodeMirror | 269 | integration/jenkins-job-builder | 252 | mediawiki/extensions/CentralNotice | 246 | mediawiki/extensions/Wikidata | 227 | mediawiki/extensions/ConfirmEdit | 225 | mediawiki/extensions/WikimediaEvents | 219 | mediawiki/extensions/Math | 215 | operations/debs/python-kafka | 214 # Incident Review 201709-201809 | Date | Service | Incident | 2017-09-17 | mediawiki | 20170917-MediaWiki-API | 2017-09-20 | elasticsearch | 20170920-Elasticsearch | 2017-09-20 | logstash | 20170920-Logstash | 2017-09-22 | mediawiki | 20170922-REST-API-alert-pageimage-update | 2017-10-01 | labvirt | 20171001-labvirt1015 | 2017-10-17 | database | 20171017-webm-audio-mediatype | 2017-10-18 | wqds | 20171018-wdqs | 2017-11-16 | wikidata | 20171116-s5-dewiki-wikidata | 2017-11-20 | ores | 20171120-Ext:ORES | 2017-11-25 | statsd | 20171125-statsd | 2017-11-28 | ores | 20171128-ORES | 2017-11-30 | wdqs | 20171130-wdqs | 2018-01-10 | swift | 20180110-swift | 2018-01-11 | mediawiki | 20180111-LegacyEncoding | 2018-01-29 | mediawiki | 20180129-MediaWiki | 2018-02-06 | phabricator | 20180206-Phabricator | 2018-02-14 | labvirt | 20180214-labvirt1008-failure | 2018-02-20 | thumbor | 20180220-thumbor | 2018-02-22 | wdqs | 20180222-wdqs | 2018-02-26 | WikibaseQualityConstraints | 20180226-WikibaseQualityConstraints | 2018-02-28 | puppetmaster | 20180228-puppetmaster | 2018-02-29 | train | 20180229-Train-1.31.0-wmf.27 | 2018-03-02 | mediawiki | 20180302-Train | 2018-03-06 | mediawiki | 20180306-MasterMergedInWMFBranch | 2018-03-12 | varnish | 20180312-Cache-text | 2018-03-14 | ores | 20180314-ORES | 2018-04-04 | mathoid | 20180404-mathoid | 2018-04-10 | database | 20180410-Deleting a page on enwiki | 2018-04-10 | routing | 20180410-Routing | 2018-04-23 | wqds | 20180423-wdqs | 2018-05-22 | mediawiki | 20180522-MediaWiki | 2018-05-24 | wikidata | 20180524-wikidata | 2018-06-06 | mail | 20180606-mx | 2018-06-15 | phabricator | 20180615-phabricator-vandalism | 2018-06-25 | wqds | 20180625-wdqs | 2018-06-26 | LoadBalancers | 20180626-LoadBalancers | 2018-06-28 | dumps | 20180628-dumps | 2018-07-04 | mathoid | 20180704-mathoid | 2018-07-05 | EventLogging | 20180705-Eventlogging-in-Hive | 2018-07-10 | mediawiki | 20180710-MediaWiki | 2018-07-11 | mail | 20180711-fundraising undeliverable mail | 2018-07-11 | kafka | 20180711-kafka-eqiad | 2018-07-17 | train | 20180717-Train | 2018-07-24 | train | 20180724-Train | 2018-07-28 | eventlogging | 20180728-eventlogging | 2018-07-31 | phabricator | 20180731-Phabricator | 2018-08-08 | network | 20180808-Network | 2018-08-21 | train | 20180821-Train | 2018-09-18 | train | 20180918-train | Service | Incident | mediawiki | 8 | wqds | 5 | train | 5 | phabricator | 3 | ores | 3 | wikidata | 2 | mathoid | 2 | mail | 2 | labvirt | 2 | EventLogging | 2 | database | 2 | WikibaseQualityConstraints | 1 | varnish | 1 | thumbor | 1 | swift | 1 | statsd | 1 | routing | 1 | puppetmaster | 1 | network | 1 | logstash | 1 | LoadBalancers | 1 | kafka | 1 | elasticsearch | 1 | dumps | 1 {F26229059} # Prioritize e2e testing gaps | Repository | Change (commits) | Maintainers | Coverage (Selenium) | Incidents | Top 15 | mediawiki/extensions | 35616 | N/A | N/A | N/A | N/A | operations/puppet | 7354 | SRE | N/A | N/A | N/A | mediawiki/core | 6288 | RelEng | JavaScript | 8 | operations/mediawiki-config | 3160 | RelEng | N/A | N/A | N/A | mediawiki/extensions/Wikibase | 2319 | WMDE Engineering | JavaScript, Ruby | mediawiki/skins | 2098 | N/A | N/A | N/A | N/A | apps/android/wikipedia | 1959 | Reading Infrastructure | N/A | N/A | N/A | phabricator/phabricator | 1532 | RelEng | N/A | N/A | 3 | operations/software/librenms | 1446 | SRE | mediawiki/extensions/VisualEditor | 1263 | Contributors > Editing team | oojs/ui | 1229 | | mediawiki/extensions/BlueSpiceExtensions | 1176 | | integration/config | 1160 | RelEng | N/A | N/A | N/A | pywikibot/core | 1101 | | N/A | N/A | N/A | mediawiki/extensions/MobileFrontend | 923 | Reading/Web/Team | Ruby | mediawiki/extensions/ContentTranslation | 915 | Language engineering | VisualEditor/VisualEditor | 856 | Contributors > Editing team | mediawiki/extensions/CirrusSearch | 688 | Search Platform | JavaScript | 1 | translatewiki | 674 | translatewiki | N/A | N/A | N/A | mediawiki/extensions/BlueSpiceFoundation | 622 | | wikimedia/fundraising/crm | 621 | Fundraising tech | mediawiki/extensions/Popups | 603 | Reading | JavaScript | mediawiki/extensions/Flow | 563 | Growth team | operations/dns | 539 | SRE | N/A | N/A | N/A | mediawiki/extensions/Translate | 510 | Language engineering | mediawiki/extensions/Echo | 504 | Growth team | JavaScript | mediawiki/extensions/DonationInterface | 494 | Fundraising tech | mediawiki/services/parsoid | 444 | Contributors > Parsing team | mediawiki/extensions/WikimediaMessages | 380 | Growth team | mediawiki/extensions/CentralAuth | 376 | Unassigned | Ruby | mediawiki/vagrant | 373 | RelEng | N/A | N/A | N/A | mediawiki/extensions/AbuseFilter | 354 | MediaWiki Platform team | mediawiki/extensions/Collection | 328 | Unassigned | mediawiki/services/restbase | 327 | Wikimedia Services team | mediawiki/extensions/UploadWizard | 311 | Reading > Multimedia team | mediawiki/extensions/FlaggedRevs | 297 | Growth team (Pending) | mediawiki/extensions/TimedMediaHandler | 291 | Readers Engineering | mediawiki/extensions/ProofreadPage | 283 | Contributors > Editing team | mediawiki/extensions/UniversalLanguageSelector | 283 | Language engineering | mediawiki/extensions/Newsletter | 276 | Unassigned | JavaScript | mediawiki/skins/Vector | 275 | Reading | mediawiki/extensions/Cite | 271 | Contributors > Editing team | mediawiki/extensions/CodeMirror | 269 | Community Tech | integration/jenkins-job-builder | 252 | RelEng | N/A | N/A | N/A | mediawiki/extensions/CentralNotice | 246 | Fundraising tech | Ruby | mediawiki/extensions/Wikidata | 227 | WMDE Engineering | | 2 | mediawiki/extensions/ConfirmEdit | 225 | Contributors > Editing team | mediawiki/extensions/WikimediaEvents | 219 | Unassigned | PHP | mediawiki/extensions/Math | 215 | Contributors > Editing team | JavaScript | operations/debs/python-kafka | 214 | SRE | N/A | N/A | 1

{F26477269} Review incidents from the past year to determine how many had a testable regression as the cause. From review identify the top 15 target projects. - [[ https://office.wikimedia.org/wiki/Release_Engineering/FY1819-Planning/Code_Health | Release Engineering/FY1819-Planning/Code Health ]] > FY1819-Q1 > From review identify the top 15 target projects. - [[ https://www.mediawiki.org/wiki/Wikimedia_Technology/Annual_Plans/FY2019/TEC13:_Code_Health/Goals#Outcome_2_/_Output_2.2 | Wikimedia Technology/Annual Plans/FY2019/TEC13: Code Health/Goals > Outcome 2 / Output 2.2 ]] - Review existing e2e test coverage. - Define prioritization scheme. - Prioritize e2e testing gaps. # Review existing e2e test coverage List of repositories with at least one Selenium test. | **Repository** | **Language** | mediawiki/core | JavaScript | mediawiki/extensions/AdvancedSearch | JavaScript | mediawiki/extensions/CentralAuth | Ruby | mediawiki/extensions/CentralNotice | Ruby | mediawiki/extensions/CirrusSearch | JavaScript | mediawiki/extensions/Echo | JavaScript | mediawiki/extensions/ElectronPdfService | JavaScript | mediawiki/extensions/GettingStarted | Ruby | mediawiki/extensions/Math | JavaScript | mediawiki/extensions/MobileFrontend | Ruby | mediawiki/extensions/MultimediaViewer | Ruby | mediawiki/extensions/Newsletter | JavaScript | mediawiki/extensions/ORES | JavaScript | mediawiki/extensions/Popups | JavaScript | mediawiki/extensions/QuickSurveys | Ruby | mediawiki/extensions/RelatedArticles | JavaScript | mediawiki/extensions/RevisionSlider | Ruby | mediawiki/extensions/TwoColConflict | JavaScript, Ruby | mediawiki/extensions/Wikibase | JavaScript, Ruby | mediawiki/extensions/WikibaseLexeme | JavaScript, Ruby | mediawiki/extensions/WikimediaEvents | PHP | mediawiki/skins/MinervaNeue | Ruby | phab-deployment | JavaScript | wikimedia/portals/deploy | JavaScript # Define prioritization scheme - How much change did happen (in the last 12 months)? - https://wikimedia.biterg.io > Last 90 days > Absolute > From 2017-11-01 00:00:00.000 > To 2018-10-31 23:59:59.999 > Go - Git > Overview > Repositories > raw: P7776 - Git > Attraction/Retention > Top Repositories > raw: P7777 - Gerrit > Overview > Repositories > raw: P7778 - Does a repository have maintainers? - https://www.mediawiki.org/wiki/Developers/Maintainers - { (See also T190891}) - Do maintainers want tests? - [[ https://docs.google.com/spreadsheets/d/1l6zyuV-vbJM1P3qHAqaMfegpECeItJeKN9ToUdl8jmk/edit?usp=sharing | Incident Review 201711-201810 ]] ## How much change did happen [[ https://wikimedia.biterg.io | Bitergia ]] report for the last year (P7586): | Repository | Commits | mediawiki/extensions | 35616 | operations/puppet | 7354 | mediawiki/core | 6288 | operations/mediawiki-config | 3160 | mediawiki/extensions/Wikibase | 2319 | mediawiki/skins | 2098 | apps/android/wikipedia | 1959 | phabricator/phabricator | 1532 | operations/software/librenms | 1446 | mediawiki/extensions/VisualEditor | 1263 | oojs/ui | 1229 | mediawiki/extensions/BlueSpiceExtensions | 1176 | integration/config | 1160 | pywikibot/core | 1101 | mediawiki/extensions/MobileFrontend | 923 | mediawiki/extensions/ContentTranslation | 915 | VisualEditor/VisualEditor | 856 | mediawiki/extensions/CirrusSearch | 688 | translatewiki | 674 | mediawiki/extensions/BlueSpiceFoundation | 622 | wikimedia/fundraising/crm | 621 | mediawiki/extensions/Popups | 603 | mediawiki/extensions/Flow | 563 | operations/dns | 539 | mediawiki/extensions/Translate | 510 | mediawiki/extensions/Echo | 504 | mediawiki/extensions/DonationInterface | 494 | mediawiki/services/parsoid | 444 | mediawiki/extensions/WikimediaMessages | 380 | mediawiki/extensions/CentralAuth | 376 | mediawiki/vagrant | 373 | mediawiki/extensions/AbuseFilter | 354 | mediawiki/extensions/Collection | 328 | mediawiki/services/restbase | 327 | mediawiki/extensions/UploadWizard | 311 | mediawiki/extensions/FlaggedRevs | 297 | mediawiki/extensions/TimedMediaHandler | 291 | mediawiki/extensions/ProofreadPage | 283 | mediawiki/extensions/UniversalLanguageSelector | 283 | mediawiki/extensions/Newsletter | 276 | mediawiki/skins/Vector | 275 | mediawiki/extensions/Cite | 271 | mediawiki/extensions/CodeMirror | 269 | integration/jenkins-job-builder | 252 | mediawiki/extensions/CentralNotice | 246 | mediawiki/extensions/Wikidata | 227 | mediawiki/extensions/ConfirmEdit | 225 | mediawiki/extensions/WikimediaEvents | 219 | mediawiki/extensions/Math | 215 | operations/debs/python-kafka | 214 # Incident Review 201709-201809 | Date | Service | Incident | 2017-09-17 | mediawiki | 20170917-MediaWiki-API | 2017-09-20 | elasticsearch | 20170920-Elasticsearch | 2017-09-20 | logstash | 20170920-Logstash | 2017-09-22 | mediawiki | 20170922-REST-API-alert-pageimage-update | 2017-10-01 | labvirt | 20171001-labvirt1015 | 2017-10-17 | database | 20171017-webm-audio-mediatype | 2017-10-18 | wqds | 20171018-wdqs | 2017-11-16 | wikidata | 20171116-s5-dewiki-wikidata | 2017-11-20 | ores | 20171120-Ext:ORES | 2017-11-25 | statsd | 20171125-statsd | 2017-11-28 | ores | 20171128-ORES | 2017-11-30 | wdqs | 20171130-wdqs | 2018-01-10 | swift | 20180110-swift | 2018-01-11 | mediawiki | 20180111-LegacyEncoding | 2018-01-29 | mediawiki | 20180129-MediaWiki | 2018-02-06 | phabricator | 20180206-Phabricator | 2018-02-14 | labvirt | 20180214-labvirt1008-failure | 2018-02-20 | thumbor | 20180220-thumbor | 2018-02-22 | wdqs | 20180222-wdqs | 2018-02-26 | WikibaseQualityConstraints | 20180226-WikibaseQualityConstraints | 2018-02-28 | puppetmaster | 20180228-puppetmaster | 2018-02-29 | train | 20180229-Train-1.31.0-wmf.27 | 2018-03-02 | mediawiki | 20180302-Train | 2018-03-06 | mediawiki | 20180306-MasterMergedInWMFBranch | 2018-03-12 | varnish | 20180312-Cache-text | 2018-03-14 | ores | 20180314-ORES | 2018-04-04 | mathoid | 20180404-mathoid | 2018-04-10 | database | 20180410-Deleting a page on enwiki | 2018-04-10 | routing | 20180410-Routing | 2018-04-23 | wqds | 20180423-wdqs | 2018-05-22 | mediawiki | 20180522-MediaWiki | 2018-05-24 | wikidata | 20180524-wikidata | 2018-06-06 | mail | 20180606-mx | 2018-06-15 | phabricator | 20180615-phabricator-vandalism | 2018-06-25 | wqds | 20180625-wdqs | 2018-06-26 | LoadBalancers | 20180626-LoadBalancers | 2018-06-28 | dumps | 20180628-dumps | 2018-07-04 | mathoid | 20180704-mathoid | 2018-07-05 | EventLogging | 20180705-Eventlogging-in-Hive | 2018-07-10 | mediawiki | 20180710-MediaWiki | 2018-07-11 | mail | 20180711-fundraising undeliverable mail | 2018-07-11 | kafka | 20180711-kafka-eqiad | 2018-07-17 | train | 20180717-Train | 2018-07-24 | train | 20180724-Train | 2018-07-28 | eventlogging | 20180728-eventlogging | 2018-07-31 | phabricator | 20180731-Phabricator | 2018-08-08 | network | 20180808-Network | 2018-08-21 | train | 20180821-Train | 2018-09-18 | train | 20180918-train | Service | Incident | mediawiki | 8 | wqds | 5 | train | 5 | phabricator | 3 | ores | 3 | wikidata | 2 | mathoid | 2 | mail | 2 | labvirt | 2 | EventLogging | 2 | database | 2 | WikibaseQualityConstraints | 1 | varnish | 1 | thumbor | 1 | swift | 1 | statsd | 1 | routing | 1 | puppetmaster | 1 | network | 1 | logstash | 1 | LoadBalancers | 1 | kafka | 1 | elasticsearch | 1 | dumps | 1 {F26229059} # Prioritize e2e testing gaps | Repository | Change (commits) | Maintainers | Coverage (Selenium) | Incidents | Top 15 | mediawiki/extensions | 35616 | N/A | N/A | N/A | N/A | operations/puppet | 7354 | SRE | N/A | N/A | N/A | mediawiki/core | 6288 | RelEng | JavaScript | 8 | operations/mediawiki-config | 3160 | RelEng | N/A | N/A | N/A | mediawiki/extensions/Wikibase | 2319 | WMDE Engineering | JavaScript, Ruby | mediawiki/skins | 2098 | N/A | N/A | N/A | N/A | apps/android/wikipedia | 1959 | Reading Infrastructure | N/A | N/A | N/A | phabricator/phabricator | 1532 | RelEng | N/A | N/A | 3 | operations/software/librenms | 1446 | SRE | mediawiki/extensions/VisualEditor | 1263 | Contributors > Editing team | oojs/ui | 1229 | | mediawiki/extensions/BlueSpiceExtensions | 1176 | | integration/config | 1160 | RelEng | N/A | N/A | N/A | pywikibot/core | 1101 | | N/A | N/A | N/A | mediawiki/extensions/MobileFrontend | 923 | Reading/Web/Team | Ruby | mediawiki/extensions/ContentTranslation | 915 | Language engineering | VisualEditor/VisualEditor | 856 | Contributors > Editing team | mediawiki/extensions/CirrusSearch | 688 | Search Platform | JavaScript | 1 | translatewiki | 674 | translatewiki | N/A | N/A | N/A | mediawiki/extensions/BlueSpiceFoundation | 622 | | wikimedia/fundraising/crm | 621 | Fundraising tech | mediawiki/extensions/Popups | 603 | Reading | JavaScript | mediawiki/extensions/Flow | 563 | Growth team | operations/dns | 539 | SRE | N/A | N/A | N/A | mediawiki/extensions/Translate | 510 | Language engineering | mediawiki/extensions/Echo | 504 | Growth team | JavaScript | mediawiki/extensions/DonationInterface | 494 | Fundraising tech | mediawiki/services/parsoid | 444 | Contributors > Parsing team | mediawiki/extensions/WikimediaMessages | 380 | Growth team | mediawiki/extensions/CentralAuth | 376 | Unassigned | Ruby | mediawiki/vagrant | 373 | RelEng | N/A | N/A | N/A | mediawiki/extensions/AbuseFilter | 354 | MediaWiki Platform team | mediawiki/extensions/Collection | 328 | Unassigned | mediawiki/services/restbase | 327 | Wikimedia Services team | mediawiki/extensions/UploadWizard | 311 | Reading > Multimedia team | mediawiki/extensions/FlaggedRevs | 297 | Growth team (Pending) | mediawiki/extensions/TimedMediaHandler | 291 | Readers Engineering | mediawiki/extensions/ProofreadPage | 283 | Contributors > Editing team | mediawiki/extensions/UniversalLanguageSelector | 283 | Language engineering | mediawiki/extensions/Newsletter | 276 | Unassigned | JavaScript | mediawiki/skins/Vector | 275 | Reading | mediawiki/extensions/Cite | 271 | Contributors > Editing team | mediawiki/extensions/CodeMirror | 269 | Community Tech | integration/jenkins-job-builder | 252 | RelEng | N/A | N/A | N/A | mediawiki/extensions/CentralNotice | 246 | Fundraising tech | Ruby | mediawiki/extensions/Wikidata | 227 | WMDE Engineering | | 2 | mediawiki/extensions/ConfirmEdit | 225 | Contributors > Editing team | mediawiki/extensions/WikimediaEvents | 219 | Unassigned | PHP | mediawiki/extensions/Math | 215 | Contributors > Editing team | JavaScript | operations/debs/python-kafka | 214 | SRE | N/A | N/A | 1