I wonder if this is failing because now there is a third survey (drink-survey) that is not being declared as dismissed. Worth someone investigating. I'm concerned there may be an issue with our sampling as that quick survey is not meant to show at all (0 sample)
You'll probably need to run this browser test locally various times until you can replicate the issue.
| Subject | Author | Repo | Branch | Lines +/- | |
|---|---|---|---|---|---|
| QA: Ensure survey code loaded before testing if survey is visible | Jdlrobson | mediawiki/extensions/QuickSurveys | dev | +17 -0 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | leila | T112326 [Epic] Reader segmentation research - Q2 | |||
| Resolved | • jhobs | T113443 First QuickSurvey for reader segmentation research - external survey | |||
| Resolved | Jdlrobson | T111445 GOAL: Run our first survey campaign | |||
| Resolved | Jdlrobson | T113534 Investigate QuickSurveys browser tests failures | |||
| Resolved | dduvall | T105589 MW-Selenium associates wrong SauceLabs job with Jenkins artifact |
Tested against beta labs and no luck @dduvall any ideas? Jenkins is not giving us much to work on with the broken Sauce Labs urls..
I really have no clue what's going on here. T105589 needs to get fixed asap to unblock us.
Change 246801 had a related patch set uploaded (by Jdlrobson):
QA: Don't ask. Just merge.
It's hard for me to comment without knowing more about the case that you're testing and at least a bit of the underlying implementation. Is there a way we can simulate the xhr latency so as to reproduce this locally?
I have created a Jenkins job to test 246801 (diff from the original job).
Results:
I still do not agree with using sleep, see T48887: Remove "sleep" from tests.
@zeljkofilipin: All I see is "No permission to view config history". Is this because you're using super secret sauce to get the test passing? ;)
@phuedx I think you have to be logged in to jenkins to see job config history, but this is the only change in the job shell script (applying the gerrit patch):
git fetch https://gerrit.wikimedia.org/r/mediawiki/extensions/QuickSurveys refs/changes/01/246801/1 && git checkout FETCH_HEAD
I think you have to be logged in to jenkins to see job config history…
Ha! Of course I do.
@Jdlrobson: The commit message of 246801 needs rewording and there's a sleep command in there that @zeljkofilipin doesn't agree with. The test does pass though…
Came back to look at the patch https://gerrit.wikimedia.org/r/246801
No rest for the wicked.
Would be great to have a review but please do not lose any sleep over it, I know this bug has been tiring.
Change 246801 merged by jenkins-bot:
QA: Ensure survey code loaded before testing if survey is visible
Closing since the last QuickSurvey browser test failure happened on December 15, a day before the above patch was merged.