Page MenuHomePhabricator

Create an outline of QA/Browser test workshops to give
Closed, ResolvedPublic

Description

Including general topics included in each one.

Chris and I had a meeting today and we talked about this. Our notes are here:

http://etherpad.wikimedia.org/p/releng

I will paste them here, in case the pad disappears:

The first meeting would be a meetup, Dan talking about vagrant/puppet. Appox. one workshop/talk a month, but we do not have to have a workshop/talk every month.

  • Vagrant and puppet internals (two or one topic? ask Dan)
    • in general
    • for browser tests
    • make the first contribution from mediawiki-vagrant machine
    • running browser tests from mediawiki-vagrant machine
      • which browsers are available?
  • Survey of practice of all tested extensions. - how stuff is implemented, tips and tricks (Chris to do the survey and collate examples for discussion)
    • example: rspec expect syntax
    • example: step re-use in MF
    • example: heuristics for identifying real bugs vs. test problems (including good ATDD descriptions and confusing ATDD descriptions)
  • Current issues and problems with tests and systems (e.g. performance; IE; Selenium vs VE; etc.)
    • Jenkins architecture and limitations
    • Cursoring in VE
    • handling iframes well
    • challenges for Flow
    • REALLY do ATDD/BDD. How can we make the tests pass before merge to master?
  • Exploratory testing
    • guest speaker? (Chris can think of a few SF people, Elisabeth Hendrickson)
    • exploratory testing vs test automation
  • Jenkins
    • Jenkins Job Builder!
    • Zuul
    • Running tests after patch set submission
    • Running tests once/twice a day
  • Test environments
    • Current state of beta labs, testwiki, test2wiki/mw.o/etc
    • Future needs
      • Second beta labs?
      • Bare metal env for performance testing?
      • Multiple hosted vagrant or docker envs?
NOTE: we should update Guillaume's categories for monthly tech news also.

Details

Reference
fl619

Event Timeline

flimport raised the priority of this task from to Normal.Sep 12 2014, 1:48 AM
flimport set Reference to fl619.

dduvall wrote on 2014-09-05 18:47:18 (UTC)

Also along the BDD lines, I would love to see an explanation of declarative vs. imperative styles and when one might be more appropriate over the other. Most of the tests I've seen use an imperative style which is typically more appropriate when writing very targeted tests (e.g. regression tests), but not necessarily good for BDD acceptance tests that are supposed to serve as documentation (user stories) as well as steps for the test runner.

Another good point to be made in the BDD discussion is how Cucumber can benefit non-engineers as collaborative product documentation.

The outline is created.

Is there something else that needs to be done here? Should we post it to mediawiki.org? Should we provide more details about each workshop?

Probably archive it somewhere on mw.org. The next steps are to schedule and do them, I suppose :)

zeljkofilipin removed zeljkofilipin as the assignee of this task.Nov 20 2014, 1:41 PM
zeljkofilipin set Security to None.
zeljkofilipin added a subscriber: zeljkofilipin.

Notes from refactoring about confusion I see:

  • Proper use of Given, When, and Then, in particularly Then
    • Then steps should always have an assertion, Given and When should not.
  • Proper re-use of steps