Customize query in gerrit
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | leila | T112326 [Epic] Reader segmentation research - Q2 | |||
| Resolved | • jhobs | T113443 First QuickSurvey for reader segmentation research - external survey | |||
| Resolved | Jdlrobson | T111445 GOAL: Run our first survey campaign | |||
| Resolved | • jhobs | T110661 Deploy QuickSurveys extension to the English Wikipedia | |||
| Declined | None | T96785 In-channel feedback with star rating / picklist (and on apps, simple text entry?) and backend de-spam | |||
| Resolved | • Jhernandez | T104439 [GOAL]: Develop a mechanism for quick surveys on desktop + mobile | |||
| Resolved | • rmoen | T110199 Get QuickSurveys enabled on beta cluster | |||
| Duplicate | None | T110194 QuickSurveys code quality | |||
| Resolved | Jdlrobson | T110198 Add basic browser tests | |||
| Resolved | • bmansurov | T110225 Add basic browser tests: external surveys |
Event Timeline
Comment Actions
Change 233933 had a related patch set uploaded (by Bmansurov):
Add browser tests for external surveys
Comment Actions
Change 233930 had a related patch set uploaded (by Bmansurov):
Add an option to load a specific survey
Comment Actions
Change 234710 had a related patch set uploaded (by Bmansurov):
Add browser tests for external surveys
Comment Actions
Change 234710 abandoned by Bmansurov:
Add browser tests for external surveys
Comment Actions
All tests pass locally when I disable the existing step "And the page has fully loaded". I'm not sure why that step doesn't work on my computer. And remotely, I get different errors. Does anyone know howto fix it? @phuedx, @rmoen, @Jdlrobson?
Comment Actions
Judging on the artifact [1] attached to the report the survey seems to be loading twice. So I'd start with investigating why that might be.
[1] https://integration.wikimedia.org/ci/job/mwext-mw-selenium/669/artifact/log/External%20surveys%3A%20External%20survey%20is%20not%20present%20without%20flag.mp4
[2] https://integration.wikimedia.org/ci/job/mwext-mw-selenium/669/console
Comment Actions
(I'm in a planning meeting all morning so will look at this later but it's possible you've just uncovered a new bug unrelated to your patch - I'd suggest you try adding the browser tests before your patch to see if the expected tests fail in those circumstance)
Comment Actions
@Jdlrobson, thanks. Either the bucketing logic seems off or I'm not understanding it correctly. Say, my example survey has 0.1 coverage, then I'm always assigned to the control bucket, even though I should be assigned to the bucket A 10% of the time. No?
Comment Actions
It's possible. Write some unit tests to test it more thoroughly. I believe in you Baha! You'll work it out :)
Comment Actions
Either the bucketing logic seems off or I'm not understanding it correctly. Say, my example survey has 0.1 coverage, then I'm always assigned to the control bucket, even though I should be assigned to the bucket A 10% of the time. No?
The current bucketing mechanism operates like "10% of users see the survey all of the 100% of the time – until they interact with it".
Comment Actions
See my review: https://gerrit.wikimedia.org/r/#/c/233933/ - issues with browser tests
Comment Actions
Thanks, I've replied to your comment, @Jdlrobson.
Thanks for the clarification, @phuedx.
Comment Actions
I can replicate the issue that Jenkins has locally. I will post a follow-up patch for your inspection when you wake up :)
Comment Actions
@bmansurov see https://gerrit.wikimedia.org/r/235618 - primarily goal here is to make the feature files as readable as possible. I hope things become clearer with this patch. Feel free to squash it/take ideas from it/whatever :)