Assigning to Monte for an initial pass.

As a first step we've created an etherpad (https://etherpad.wikimedia.org/p/iOS-regression-script) with a proposed re-org of the test sections. We aimed to group things in a way that represents the real uses of the app, and ordered them to prioritize early and common paths (new installs, article reading) and put less common actions and paths lower.

The next step will be to move the existing test items into the new structure and make sub-cases more atomic and their steps less ambiguous.

We touched on this a bit in the meeting but we should explicitly test the upgrade path. And I would suggest not having a set of tests for the upgrade path, but instead adding a column for running the entire regression suite from the upgrade path.

We don't need to do this for every platform OS, but I would suggest the current OS on the iPhone platform. It's not perfect coverage but its something better than we have now.

So it would be:
iPhone Upgrade, iOS 9, iPhone clean install iOS 9, iPhone iOS 8.4, iPad iOS 9, iPad 8.4

@JMinor @Fjalapeno
In trying to cross-reference our proposed new structure ( https://etherpad.wikimedia.org/p/iOS-regression-script ) with the existing regression script's items, I found our proposed structure to be lacking, so I fleshed it out even further. It still needs some work, but I'd like to do that work, and I'd like to propose the following next steps:

1. Josh and I hangout with Andy to see how open they are to sudden dramatic changes to the regression script and potentially its document type.
   • Unknowns
     1. Is TSG open to us moving our regression scripts to wiki page(s) instead of the spreadsheet?
     2. If so, is there a small portion of the testing suite we could try initially as an experimental spike?
        1. Questions to answer during spike: Are there ways for the TSG folks, or other testers, to replicate their spreadsheet workflow? i.e. on spreadsheet a tab is created, and as each test is run the tester marks "pass", "fail" etc and sometimes adds notes. Are there wiki features we could use to support this?
     3. Is it problematic for TSG's rhythm if we completely change, in one swoop, the entire organizational structure of the document, whether spreadsheet, or wiki?
     4. If we move to a wiki, could we break it up so we have one wiki page per section (or per test?) so the document isn't so monolithic - it would be easier to add things such as animated gif and other clarifications if we're not dealing with a single giant doc.
2. I would like to continue to flesh out the high level proposed re-org of our test sections (https://etherpad.wikimedia.org/p/iOS-regression-script)
   1. Continue adding high level sections for all parts of the app for which we don't have good regression coverage - i.e. Settings or some of the language features.
   2. Continue refining the re-org to make running the complete regression "flow" more naturally with less duplication and fewer context switches - i.e. more deliberate grouping of RTL language tests, or article tests, etc.
3. Take the high level organization and coverage improvements from Step 2 and do one of the following depending on what we learn in Step 1:
   1. Either re-make and re-organize the spreadsheet to reflect the high level proposed re-org of our test sections (https://etherpad.wikimedia.org/p/iOS-regression-script)
   2. - OR -
   3. Implement a wiki-fied fusion of the spreadsheet and the proposed changes here: (https://etherpad.wikimedia.org/p/iOS-regression-script)

I am willing to, say, devote my next few Fridays to this as part of my wider goal of improving testing, if that sounds ok. My goal is *much* wider regression coverage, easier regression run-throughs, improved ease of maintenance, and especially if we can move these on-wiki potentially getting community help.

@Mhurd looks like a good plan

I don't think this is something we can improve drastically by Tuesday.

Lets go over this next week with @JMinor
And get his thoughts.

@Mhurd Looks great at a glance. Probably need to talk with Andy about this, not just Nicholas. Might be worth setting up a separate meeting.

https://phabricator.wikimedia.org/T143053#2676135