Page MenuHomePhabricator

Rewrite MobileFrontend and Page previews automated browser tests in Node.js
Closed, DuplicatePublic13 Estimated Story Points

Description

After October Ruby browser tests will no longer be maintained. We will need to port all our extensions browser tests to be written in Node. The bulk of our browser tests are written in MinervaNeue so to give us practice and prep for these we will start with the smaller extensions which should be much more straightforward. I suggest this may be best addressed by some pair programming where 2 people work on 1 extension together (and we alternate pairings).

The parent task mentions MinervaNeue (tracked in T174018) as well.
I believe we will be more efficient if we address MobileFrontend, QuickSurveys, Page previews together rather than one at a time. I see this as an opportunity for everyone in the team to invest time and get up to speed with how this works. The number of browser tests in these extensions is low: QuickSurveys has 12, Popups 2, MobileFrontend has 13 (Minerva has 75 for comparison)

After discussion, we questioned the value of QuickSurveys and will delay the decision there as we might either remove those browser tests altogether or delay the rewrite.

Precursors:

  • Related pages browser tests have been fully written in Node.js (see T164024)

Acceptance criteria

  • Enable running of browser tests in Node via check experimental for MobileFrontend, QuickSurveys and Page Previews
  • Rewrite browser tests for the 2 extensions in a new selenium folder
  • When tests are passing, switch to new job
  • Delete the tests/browser folder
  • While working on this, do an audit as originally proposed in T148973 of what we are testing. Remove unnecessary tests and document missing tests. At the end, summarise the browser test for each extension.

Sign off notes

  • The setup of the daily builds will come afterwards (see T171847). Create a task to do this for the 3 extensions (or if you are tech lead make that happen).

Related Objects

StatusSubtypeAssignedTask
OpenNone
ResolvedJdlrobson
DeclinedNone
Resolvedzeljkofilipin
Resolvedzeljkofilipin
DuplicateJdlrobson
ResolvedJdlrobson
Resolvedzeljkofilipin
Resolvedzeljkofilipin
OpenNone
ResolvedJdlrobson
ResolvedNone
Resolvedawight
DuplicateLegoktm
StalledNone
OpenNone
Resolvedzeljkofilipin
Resolvedhashar
OpenNone
ResolvedPRODUCTION ERRORJdlrobson
ResolvedJdlrobson
ResolvedJdrewniak
OpenNone
OpenEdtadros
OpenNone
Resolvedzeljkofilipin
ResolvedEdtadros
OpenNone
ResolvedJdlrobson

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

I'm moving this back to Triaged but Future.

^ Context: There's an ongoing discussion around porting existing automated browser tests to Node. IIRC we have ~6 months to do so before the existing tests are no longer supported by Release-Engineering-Team.

Jdlrobson added a subscriber: Jdlrobson.

Jon to flesh out the description before next sprint. It's not fair on implementer to have to go through the history of this task.

@phuedx also pointed out we might want to not do this and instead do the port to Node.js straight away. The latter is going to take a lot longer. @zeljkofilipin is there anything we should keep in mind about taking this approach. Will not upgrading to Selenium 3 while we migrate away cause us any problems?

You can use ruby+selenium 2 for as long as we support ruby+selenium. (Roughly for the next 6 months.) I am available for about 50% of my time to help with the migration to node+selenium. To make it explicit, if you would like to pair with me for several hours every day on writing node+selenium tests, I am available.

Jdlrobson renamed this task from Upgrade MobileFrontend automated browser tests to Selenium 3 to Rewrite MobileFrontend automated browser tests in Node.js.Apr 27 2017, 5:57 PM
Jdlrobson updated the task description. (Show Details)
Jdlrobson changed the task status from Open to Stalled.Apr 27 2017, 6:04 PM

Stalled until T164024 is resolved

Still stalled unfortunately.

Jdlrobson renamed this task from Rewrite MobileFrontend automated browser tests in Node.js to Rewrite MobileFrontend, QuickSurveys, Page previews automated browser tests in Node.js.Aug 24 2017, 1:45 PM
Jdlrobson changed the task status from Stalled to Open.
Jdlrobson removed a project: MobileFrontend.
Jdlrobson updated the task description. (Show Details)
Jdlrobson moved this task from Needs Prioritization to Upcoming on the Readers-Web-Backlog board.

RelatedArticles shows how this is done. We'll need to upgrade sooner rather than later as otherwise our existing Ruby tests will be unsupported.

To make things easier before we commit to working on this, I can do the setup of check experimental and the Node.js environment - hopefully with help from @zeljkofilipin

No it's a subtask.

phuedx added a comment.EditedAug 24 2017, 3:08 PM

No it's a subtask.

That's true given their current relationship. If you compare both tasks directly, however, they do look like duplicates.

Should there be a task per extension, which the parent epic then links to in the description?

Jdlrobson added a comment.EditedAug 24 2017, 3:21 PM

That's true given their current relationship. If you compare both tasks directly, however, they do look like duplicates.

The parent task mentions MinervaNeue (tracked in T174018) as well. If it's easier to think of these as two separate tasks we can forget about the epic, but I find the umbrella task useful in case we decide to descope this task. I believe we will be more efficient if we address MobileFrontend, QuickSurveys, Page previews together rather than one at a time. We only have 2 months to do this, so I see this as an opportunity for everyone in the team to invest time and get up to speed with how this works. The number of browser tests in these extensions is low: QuickSurveys has 12, Popups 7, MobileFrontend has 13 (Minerva has 75 for comparison)

That's true given their current relationship. If you compare both tasks directly, however, they do look like duplicates.

We only have 2 months to do this, so I see this as an opportunity for everyone in the team to invest time and get up to speed with how this works.

Per https://lists.wikimedia.org/pipermail/engineering/2017-August/000464.html, we have more than 2 months to do this.

Is there value in breaking out three subtasks, one for each extension?

Is there value in breaking out three subtasks, one for each extension?

My recommendation is to not do this per https://phabricator.wikimedia.org/T160238#3549161

Change 375384 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/Popups@master] WIP: Port Popups browser tests to selenium

https://gerrit.wikimedia.org/r/375384

Jdlrobson updated the task description. (Show Details)Sep 1 2017, 2:37 PM

Is there value in breaking out three subtasks, one for each extension?

I see the following benefits to breaking this task up:

  • Smaller tasks (smaller in scope and complexity) are more likely to make it into sprints.
    • I write this acknowledging the risky, large-scoped tasks we've brought into our sprints recently. I feel that smaller tasks can be used as "padding".
  • There's less risk: say we commit to porting the QuickSurveys browser tests and we find out that we simply can't do X. If this were to impact porting the MobileFrontend browser tests, then we could fold that risk into the estimate for the MobileFrontend-related task. This would make the task clearer for us and for our stakeholders.
  • The individual tasks could be worked on in parallel with less coordination {{Citation needed}}.

Change 375377 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[integration/config@master] Add Jenkins job for browser tests

https://gerrit.wikimedia.org/r/375377

Change 375384 abandoned by Jdlrobson:
WIP: Port Popups browser tests to selenium

https://gerrit.wikimedia.org/r/375384

phuedx added a comment.Oct 3 2017, 1:54 PM

Given that there are now subtasks for each extension, I think that the answer is now "Yes." /cc @Jdlrobson

Jdlrobson updated the task description. (Show Details)Oct 3 2017, 2:01 PM

No. This is still not a dupe. I'm still pushing for us to consider these three extensions in one go but have opened subtasks so we can consider the risks of doing them separately one at a time.

When discussing this, we questioned the value of QuickSurveys. I'll update the task later today...

Jdlrobson renamed this task from Rewrite MobileFrontend, QuickSurveys, Page previews automated browser tests in Node.js to Rewrite MobileFrontend and Page previews automated browser tests in Node.js.Oct 3 2017, 4:37 PM
Jdlrobson updated the task description. (Show Details)
Jdlrobson set the point value for this task to 13.

Change 375377 abandoned by Zfilipin:
Add Jenkins job for browser tests

Reason:
Implemented as two separate commits.

https://gerrit.wikimedia.org/r/375377

Change 375384 restored by Jdlrobson:
WIP: Port Popups browser tests to selenium

https://gerrit.wikimedia.org/r/375384

Change 375384 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/Popups@master] Port Popups browser tests to selenium

https://gerrit.wikimedia.org/r/375384