A common need is to enable a PHP feature flag for some tests, but not for others. This might be used to configure the server for all tests in an extension, or to run some tests with and some without a feature flag.

Here's a mini-proposal for how we might implement this. It feels hopelessly complicated, feedback would be appreciated.

• Introduce a new, per-repo, JSON config file for browser tests. It defines test groups:
  • Optional list of test filename patterns. Matching tests belong to this group. One group can omit the filename pattern, this will be the default group. If a file matches multiple groups, trigger an error. If a file doesn't match any group and no default group exists, trigger an error.
  • Path to LocalSettings.php snippet which should be applied for this group's tests, and removed afterwards.
  • Alternatively, a map of wg- config variables to values. This can be used to automatically generate or augment the LocalSettings.php snippet.l