Change 230139 had a related patch set uploaded (by Dduvall):
Decouple Environment#test_name

https://gerrit.wikimedia.org/r/230139

Change 230140 had a related patch set uploaded (by Dduvall):
Decouple check for MediaWiki extension dependencies

https://gerrit.wikimedia.org/r/230140

Change 230230 had a related patch set uploaded (by Dduvall):
Decouple screenshot-ing and artifacts from Cucumber hooks

https://gerrit.wikimedia.org/r/230230

Change 230234 had a related patch set uploaded (by Dduvall):
Decouple session annotation from Cucumber

https://gerrit.wikimedia.org/r/230234

Change 230261 had a related patch set uploaded (by Dduvall):
Basic RSpec support

https://gerrit.wikimedia.org/r/230261

Hurray for a successful spike.

All the core test setup/teardown implementation has been decoupled from Cucumber and a basic harness has been implemented for RSpec—the only feature lacking is the check for MW extension dependencies but the value of those checks have always been a bit dubious anyhow.

From here, we should probably do some experimentation with teams/individuals writing browser tests. @Jdlrobson, I think your feedback on this alternative form would be tremendously valuable. Perhaps would could pair on it soon.

As an example of how this new form would look, the following is a port of some of the Cucumber scenarios from core's login.feature (this is the whole shebang, there are no separate description/implementation files):

Using the standard RSpec DSL:

describe 'login page' do
  describe 'login form' do
    it 'includes username/password fields, and a login button' do
      visit(LoginPage) do |page|
        expect(page.username_element).to be_visible
        expect(page.password_element).to be_visible
        expect(page.login_button_element).to be_visible
      end
    end

    it 'shows an error when logging in without credentials' do
      visit(LoginPage) do |page|
        page.login_element.click

        expect(page.error_box_element).to be_visible
      end
    end

    it 'shows an error when logging in with bad credentials' do
      visit(LoginPage) do |page|
        page.login_with('some bad username', 'some bad password')

        expect(page.error_box_element).to be_visible
      end
    end
  end
end

Another option is to use feature/scenario syntax (this isn't native to RSpec, but implemented by simply aliasing describe as feature, before as background, and it as scenario):

feature 'Logging in' do
  scenario 'Form contains username/password fields, and a login button' do
    visit(LoginPage) do |page|
      expect(page.username_element).to be_visible
      expect(page.password_element).to be_visible
      expect(page.login_button_element).to be_visible
    end
  end

  scenario 'Logging in without credentials shows an error' do
    visit(LoginPage) do |page|
      page.login_element.click

      expect(page.error_box_element).to be_visible
    end
  end

  scenario 'Logging in with bad credentials shows an error' do
    visit(LoginPage) do |page|
      page.login_with('some bad username', 'some bad password')

      expect(page.error_box_element).to be_visible
    end
  end
end

The latter form is closer to what we do now with Cucumber, but I personally prefer the first form as I think it promotes more descriptive examples—perhaps this is just because I'm used to writing it.

We'll also want to separate out the Cucumber and RSpec harnesses into separate gems: Fewer dependencies for each and fewer dependency conflicts (Cucumber 1.x forces us to still use RSpec 2.x).

Added this task to Testing Initiative 2015. It isn't quite "using another language" for browser testing, but it does constitute a different form/syntax for tests in Ruby.

Change 230139 merged by jenkins-bot:
Decouple Environment#test_name

https://gerrit.wikimedia.org/r/230139

Change 230140 merged by jenkins-bot:
Decouple check for MediaWiki extension dependencies

https://gerrit.wikimedia.org/r/230140

Change 230230 merged by jenkins-bot:
Decouple screenshot-ing and artifacts from Cucumber hooks

https://gerrit.wikimedia.org/r/230230

Change 230234 merged by jenkins-bot:
Decouple session annotation from Cucumber

https://gerrit.wikimedia.org/r/230234

Change 230261 merged by jenkins-bot:
Basic RSpec support

https://gerrit.wikimedia.org/r/230261