Page MenuHomePhabricator

CI tests disabled: Steps that require login are disabled and should be restored.
Closed, ResolvedPublic5 Story Points

Description

All patches to MobileFrontend were blocked by an issue with CI - several browser tests which attempt to login to the mobile site failed either freezing on the login screen or not updating after a login. This indicated a problem with the login workflow. As a result we disabled these tests but now we need to restore them.

The problem is impacting both Minerva and MobileFrontend.

Here's an example of a failure:
https://integration.wikimedia.org/ci/job/mwselenium-quibble-docker/10791/consoleFull

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 2 2019, 7:20 PM

Change 500817 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] Revert "MobileContext: Use correct method name setCookie() lieu of setcookie()"

https://gerrit.wikimedia.org/r/500817

Jdlrobson triaged this task as Unbreak Now! priority.Apr 2 2019, 7:37 PM

Seems to be blocking over extensions/skins e.g. Vector.. argggg

Restricted Application added subscribers: Liuxinyu970226, TerraCodes. · View Herald TranscriptApr 2 2019, 7:37 PM

Change 500817 abandoned by Jdlrobson:
Revert "MobileContext: Use correct method name setCookie() lieu of setcookie()"

Reason:
Okay so this is likely a change in core/infrastructure.

https://gerrit.wikimedia.org/r/500817

Jdlrobson added a subscriber: greg.

The "I am logged into the mobile website" step seems to be defined inside the mediawiki selenium gem and that's what's failing here, so there's little i can do on the MobileFrontend level. @greg will need help from your team to debug this.

Jdlrobson updated the task description. (Show Details)Apr 2 2019, 8:26 PM
greg added a subscriber: zeljkofilipin.

The "I am logged into the mobile website" step seems to be defined inside the mediawiki selenium gem and that's what's failing here, so there's little i can do on the MobileFrontend level. @greg will need help from your team to debug this.

@zeljkofilipin can you take a look when you're online, please.

From what I can see the Ruby log_in method no longer works. Maybe there was an api/security fix?

https://gerrit.wikimedia.org/r/500830

Change 500860 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] Disable browser tests on @integration with login steps

https://gerrit.wikimedia.org/r/500860

Change 500861 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] QA: Restore login steps

https://gerrit.wikimedia.org/r/500861

Change 500889 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] QA: Amend log in/using mobile site steps

https://gerrit.wikimedia.org/r/500889

Jdlrobson lowered the priority of this task from Unbreak Now! to High.Apr 3 2019, 3:55 AM

Hey @zeljkofilipin I've found a way to get them passing https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/MobileFrontend/+/500861 (disabling them) so this is no longer broken and UBN, but I would appreciate help getting these enabled again and an explanation of what happened here, so that it doesn't happen again. I've lost a whole day of development activity due to this (and am spending my evening still troubleshooting) and I'd like to understand how we can avoid these problems in future.

Change 500860 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Disable browser tests on @integration with login steps

https://gerrit.wikimedia.org/r/500860

Change 501028 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Disable some essential tests because no other options

https://gerrit.wikimedia.org/r/501028

I'm disabling these tests as a short term measure, but that's not an acceptable long term solution.

So I've disabled quite a few tests (over 20) on Minerva and MobileFrontend so the builds pass and new patches are unblocked. The underlying issue seems to be the log_in step not working any more. It works fine for me when run locally but against the beta cluster I get " unexpected HTTP response (403) (MediawikiApi::HttpError)" when logging in. However, the integration tests are still ticking along (https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-MinervaNeue/)

I'd like to get those tests back in a reasonable timeline as the logged in browser tests are pretty important.

https://gerrit.wikimedia.org/r/500861 is the patch I've been trying to enable these again with. I'd hope fixing the offending Ruby step in these would be relatively straightforward once we work out what changed.

That aside, I'm also open to using this problem to help prioritise porting the MobileFrontend/Minerva tests to Node.js which from my perspective is blocked on T199116. At minimum we need to get Ruby and Node.js browser tests running in both these repos side by side (T190710) and maybe this is a better use of our time in lieu of a quicker solution (even if it involves some kind of compromise on the how).

Zeljko are you free after the planning meeting tomorrow? If so maybe we could meet then.

Change 501028 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] QA: Disable some essential tests because no other options

https://gerrit.wikimedia.org/r/501028

So I've disabled quite a few tests (over 20) on Minerva and MobileFrontend so the builds pass and new patches are unblocked. The underlying issue seems to be the log_in step not working any more. It works fine for me when run locally but against the beta cluster I get " unexpected HTTP response (403) (MediawikiApi::HttpError)" when logging in. However, the integration tests are still ticking along (https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-MinervaNeue/)
I'd like to get those tests back in a reasonable timeline as the logged in browser tests are pretty important.
https://gerrit.wikimedia.org/r/500861 is the patch I've been trying to enable these again with. I'd hope fixing the offending Ruby step in these would be relatively straightforward once we work out what changed.

Ruby framework has been deprecated since October 2017 (J79). I don't think it would be a good use of anybody's time to debug failing Ruby tests.

That aside, I'm also open to using this problem to help prioritise porting the MobileFrontend/Minerva tests to Node.js which from my perspective is blocked on T199116. At minimum we need to get Ruby and Node.js browser tests running in both these repos side by side (T190710) and maybe this is a better use of our time in lieu of a quicker solution (even if it involves some kind of compromise on the how).

Agreed. It might take me a few weeks to resolve T199116 (since I'm not really familiar with python/quibble/docker), but I think that is the blocker for a few tasks, not just this one.

@greg should I focus on T199116 for the next few weeks? I might need a bit of @hashar's help with that.

phuedx added a subscriber: phuedx.Apr 5 2019, 5:05 PM

Change 501792 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Migrate editor_wikitext_saving.feature from Ruby to Node

https://gerrit.wikimedia.org/r/501792

Change 500889 abandoned by Jdlrobson:
QA: Amend log in/using mobile site steps

https://gerrit.wikimedia.org/r/500889

Change 502349 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] QA: Port Notifications browser test to Node.js

https://gerrit.wikimedia.org/r/502349

Jdlrobson renamed this task from CI failing: Steps that require login are failing to CI tests disabled: Steps that require login are disabled and should be restored..Apr 9 2019, 4:11 PM
Jdlrobson updated the task description. (Show Details)
ovasileva set the point value for this task to 5.Apr 9 2019, 4:12 PM
hashar removed a subscriber: hashar.Apr 9 2019, 6:18 PM

Change 502593 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Restore category step for CI

https://gerrit.wikimedia.org/r/502593

Change 502615 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Setup the selenium daily job

https://gerrit.wikimedia.org/r/502615

Change 502615 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Setup the selenium daily job

https://gerrit.wikimedia.org/r/502615

Change 502593 abandoned by Jdlrobson:
Restore category step for CI

Reason:
not working as expected

https://gerrit.wikimedia.org/r/502593

Change 502659 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] Disable broken Ruby browser tests

https://gerrit.wikimedia.org/r/502659

Change 502659 merged by Jdlrobson:
[mediawiki/extensions/MobileFrontend@master] Disable broken Ruby browser tests

https://gerrit.wikimedia.org/r/502659

Change 502822 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Beta mode cookie must be set on mobile domain

https://gerrit.wikimedia.org/r/502822

Change 502822 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Beta mode cookie must be set on mobile domain

https://gerrit.wikimedia.org/r/502822

Change 500861 abandoned by Jdlrobson:
QA: Restore login steps

Reason:
I've spent all this week fighting the Node.js framework and trying to get it to work for Minerva/MF. The Ruby framework seems to be unusable now and the mobile site is without a lot of the protection it's been enjoying. Any help appreciated getting these patches operational would be appreciated:
In particular the config patches:
https://gerrit.wikimedia.org/r/#/projects/integration/config,dashboards/default

https://gerrit.wikimedia.org/r/500861

Change 425422 had a related patch set uploaded (by Jdlrobson; owner: Jdrewniak):
[mediawiki/skins/MinervaNeue@master] Adding "diff.feature" selenium test

https://gerrit.wikimedia.org/r/425422

Change 502625 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Port remaining @login tests to Node.js

https://gerrit.wikimedia.org/r/502625

Change 425422 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Adding "diff.feature" selenium test

https://gerrit.wikimedia.org/r/425422

Change 502593 restored by Krinkle:
Restore category step for CI

https://gerrit.wikimedia.org/r/502593

Change 501792 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Migrate editor_wikitext_saving.feature from Ruby to Node

https://gerrit.wikimedia.org/r/501792

Change 502349 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] QA: Port Notifications browser test to Node.js

https://gerrit.wikimedia.org/r/502349

Change 502625 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Port remaining @login tests to Node.js

https://gerrit.wikimedia.org/r/502625

From what I can see all browser tests are now restored.
A patch remains: https://gerrit.wikimedia.org/r/502593 however that's not essential for calling the restoring of these tests complete.

selenium-daily-beta-MobileFrontend is consistently failing. @Jdlrobson, should that block resolution of this task?

Yep. I'm getting twice daily failure emails as a reminder!

I think https://gerrit.wikimedia.org/r/506474 should get them green again. Fixing these is my priority today.

Yuck! Ok, I'm going to bump this back to needs more work.

Change 502593 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Restore category step for CI

https://gerrit.wikimedia.org/r/502593

Jdlrobson closed this task as Resolved.May 1 2019, 8:39 PM
Jdlrobson claimed this task.

Tests are now green again and the remaining work is tracked in T177260 💪

mmodell changed the subtype of this task from "Task" to "Production Error".Wed, Aug 28, 11:07 PM