Page MenuHomePhabricator

QA: Setup browser tests on beta cluster so we can share test articles
Closed, ResolvedPublic2 Story Points

Description

We do not currently have a nightly job that runs on http://integration.wikimedia.org
We should set this up.
This will ensure that we can all refer to the same sample articles on beta cluster that we use for tests.
You may need help from the release engineering team to make this happen.

Event Timeline

Jdlrobson raised the priority of this task from to Needs Triage.
Jdlrobson updated the task description. (Show Details)
Jdlrobson added subscribers: Aklapper, Jdlrobson.
greg renamed this task from QA: Setup browser tests on beta labs so we can share test articles to QA: Setup browser tests on beta cluster so we can share test articles.Sep 10 2015, 11:06 PM
greg updated the task description. (Show Details)
greg set Security to None.
KLans_WMF edited a custom field.Sep 14 2015, 4:48 PM
KLans_WMF moved this task from Needs Analysis to To Do on the Reading-Web-Sprint-56-Four Lions board.
KLans_WMF triaged this task as Normal priority.Sep 14 2015, 5:14 PM
Jdlrobson moved this task from To Do to Doing on the Reading-Web-Sprint-56-Four Lions board.

Change 238974 had a related patch set uploaded (by Jdlrobson):
Setup QuickSurveys jenkins job

https://gerrit.wikimedia.org/r/238974

Change 238981 had a related patch set uploaded (by Jdlrobson):
Change default quick survey definitions

https://gerrit.wikimedia.org/r/238981

phuedx added a subscriber: phuedx.Sep 17 2015, 9:04 AM

As I commented on 238981, I'm not too happy with having example surveys enabled by default.

Change 238974 merged by jenkins-bot:
Setup QuickSurveys jenkins job

https://gerrit.wikimedia.org/r/238974

See my comment. I agree I'm just not sure if that's possible in the mediawiki/config repo - it's all a bit of a mystery to me.

@hashar see discussion above. So essentially there is some configuration in tests/browser/LocalSettings.php that we need to get on beta labs. Is there a way we can include configuration settings in mediawiki/config to make this happen?

Change 239158 had a related patch set uploaded (by Jdlrobson):
Replicate browser test config

https://gerrit.wikimedia.org/r/239158

Change 238981 abandoned by Jdlrobson:
Change default quick survey definitions

Reason:
After talking to Sam I did this instead: https://gerrit.wikimedia.org/r/239158

https://gerrit.wikimedia.org/r/238981

@Legoktm can you reply to my comment? I'm not sure what I'm supposed to do here and don't understand what's wrong.

Jdlrobson raised the priority of this task from Normal to High.Sep 18 2015, 3:43 PM

(Upping priority given we are sending spamming emails due to failures of browser tests and product/design want to review)

phuedx added a comment.EditedSep 21 2015, 8:52 AM

239158 has a +1 from me – and I've also learned a little more about the execution order of our configuration scripts (!)

@phuedx think you can +2 it or find someone who can? Really keen to get this deployed sooner rather than later so I can sign off and allow Anne some time to raise bugs...

Change 239158 merged by jenkins-bot:
Replicate browser test config for QuickSurveys

https://gerrit.wikimedia.org/r/239158

From my email reply to the latest nightly build failures:

I merged the test survey configuration and it was immediately deployed to the Beta Cluster. I then re-ran the integration tests and all but one passed [0]. Note that two failures were reported but only one was a genuine failure, which appears to be another instance of miscommunication between Jenkins and Sauce Labs.

[0] https://integration.wikimedia.org/ci/job/browsertests-QuickSurveys-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/9/testReport/(root)/External%20surveys/User_can_participate_in_external_survey/

I've moved this into -1 (Needs More Work) per T112204#1665724.