Page MenuHomePhabricator

Pick a test runner (our own or an existing one)
Closed, ResolvedPublic

Description

Candidates:

  • Phester
  • strst
  • tavern
  • behat
  • codeception
  • cypress.io
  • RobotFramework
  • SoapUI
  • dredd

Criteria:

  • runtime
  • execution model
  • ease of running locally
  • easy of running in CI
  • test language
  • ease of editing
  • easy of migration
  • scope/purpose fit
  • stability/support
  • cost to modify/maintain
  • control over development
  • documentation
  • license model
  • license sympathy
  • recursive body matches
  • regex matches
  • variables
  • variable injection
  • global fixtures
  • JSON output
  • scan for test files
  • filter tests by tag
  • parallel execution

Related Objects

StatusSubtypeAssignedTask
StalledNone
ResolvedCCicalese_WMF
Resolveddaniel
Resolveddaniel
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
ResolvedClarakosi
Resolveddaniel
ResolvedClarakosi

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 15 2019, 8:07 PM
CCicalese_WMF triaged this task as Medium priority.Jul 16 2019, 3:41 AM
CCicalese_WMF closed this task as Resolved.Aug 12 2019, 6:49 PM
CCicalese_WMF claimed this task.

Decision made: we will not build Phester. We'll use SuperTest and Mocha instead. Rationale to be documented on https://www.mediawiki.org/wiki/Core_Platform_Team/Decisions_Architecture_Research_Documentation/Using_SuperTest_Rather_Than_Creating_an_API_Integration_Test_Runner