Page MenuHomePhabricator

Getting Login Session Error when running tests with Cypress
Open, MediumPublic

Description

Status

  • we used to have the same problem with puppeteer, resolved by running each test in a separate tab
  • webdriverio, puppeteer and cypress are running the same tests, only cypress is failing

TODO

  • create a page with mediawiki in an iframe and try to log in, check if you're getting the error message
  • move code from 601375
    • to the first separate patch, make sure each test is in a separate file
    • the second separate patch, try not using mwbot and see if that fixes the problem (this patch doesn't have each test is in a separate file)
    • optionally, create a third separate patch, combining the two above patches (each test is in a separate file, not using mwbot)

When I try to Login or Create a new Account with Cypress, I keep getting the error shown in the picture below.

The tests for logging in an account work fine locally using Mediawiki-Docker but fail with this error on the CI.

The same error message at in T258121: Logging in to a wiki sometimes fails with 'sessionfailure' error (coinciding with SameSite rollout).

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptApr 5 2020, 3:11 PM

It would be useful if you said how you've resolved the problem. ๐Ÿ˜

So I didn't do anything to fix this earlier. I moved my cypress folder to the MediaWiki/tests folder and I didn't get the error anymore.

I submitted a patch. Everything works fine locally, but on the CI, the third test fails with this error.

The first test creates an account, the second tests make sure a user can log out after creating an account, and the third test ensures that a user can log in with the username and password he recently used to create a new account.

The third test which is the Login test fails on the CI with this error, a recording of the test can be found here

Hey @Gbahdeyboh this seems to be an issue with an environment variable on the CI. I had encountered a similar issue as well and maybe it has got to do with us clearing browser cookies....though not sure

In T249450#6053079, @AlQaholic007 wrote:

Hey @Gbahdeyboh this seems to be an issue with an environment variable on the CI. I had encountered a similar issue as well and maybe it has got to do with us clearing browser cookies....though not sure

Hi @AlQaholic007, I doubt if it's an issue with the environment variable because the account creation test passes.

Also, for a new user visiting the page, the browser cookies are supposed to be empty by default.

Y

In T249450#6053079, @AlQaholic007 wrote:

Hey @Gbahdeyboh this seems to be an issue with an environment variable on the CI. I had encountered a similar issue as well and maybe it has got to do with us clearing browser cookies....though not sure

Hi @AlQaholic007, I doubt if it's an issue with the environment variable because the account creation test passes.

Also, for a new user visiting the page, the browser cookies are supposed to be empty by default.

Hmmm default behaviour should be cookie veing empty. What this error seems to be pointing at however seems to be an abruptly ended session and that's maybe what's triggering a security flag?

Sometimes cypress runs too fast. Can you try to add an await between the tests and see if they work?

Hhhmmm, session hijacking seems more like we're trying to create a new session while one session hasn't ended.

In T249450#6053168, @AlQaholic007 wrote:

Sometimes cypress runs too fast. Can you try to add an await between the tests and see if they work?

The issue is, I don't get this error when running the tests locally. It only comes up on CI.

Cypress also waits for a page to completely load before executing the next tests scripts, so I don't think it's cypress running too fast.

I guess try to insert a delay see if that works?

In T249450#6053190, @AlQaholic007 wrote:

I guess try to insert a delay see if that works?

I'll try this out now and see if it works.

I've tried to make some changes but still get the same error.

I read a bit about session Hijacking and figured out that it's an attack that is used to steal a browser session (cookies or tokens) and use this session to gain authorized access to the user's account. For some reasons, the security measures put in place thinks we're trying to hijack the browsers session. I think this happens to be a cypress issue since this error only occurs on puppeteer. We might need someone from the team to tell us under what conditions does the app think it's session is being hijacked so we know what to do specifically to solve it.

are you quitting the browser between tests or using the same instance?

are you quitting the browser between tests or using the same instance?

The same Instance,

I create an account with some details, after account creation, I log out, then clear browser cookies and click the login button so I can test If I can Login with the details of the newly created account. It's the login that gives this error.

Everything works fine locally, but only the login test fails on CI.

try without using the same instance.
write different tests without clearing browser and cookies.

try without using the same instance.
write different tests without clearing browser and cookies.

I'm not sure how to do that on cypress, cypress doesn't not support closing the browser window with a command.

I don't understand the logic in your tests.
you shouldn't need to clear any cookies.
on patch 12 the tests are failing on my machine.

with this changes the tests work locally without any async or awaits nor clearing cookies, change ur code accordingly and make sure the tests pass locally and then push to see if this fixes the issue in the CI.
try to delete all the clear cookies and storage code and do:
(pseudo code)

//file-spec.js
it( 'Should create account', () => {
   //create account
  //verify account created
} );

it( 'Should log out', () => {
  //log out
  //verify logout
} );

it( 'Should  login ', () => {
  //login
  //verify login
} );

cypress handles the state of the app automatically between tests.
if you want cypress to close the browser and open another one just create multiple spec files with different tests inside, this way the browser is closed and all cookies and storage is new

(pseudo code)

//createAccount-spec.js
it( 'Should create account', () => {
   //create account
  //verify account created
} );

//logout-spec.js
it( 'Should log out', () => {
   //create account
  //verify login
  //log out
  //verify logout
} );

//login-spec.js
it( 'Should be able to login after logout', () => {
  //create account
  //verify login
  //log out
  //verify logout
  //login
  //verify login
} );

in order to avoid issues with all the times u create accounts on the 2nd method, you should use the media api to create account and get the cookie or token and set it on the browser so ur already loggedin.

all this information is in the documentation of cypress and examples , I really suggest that before trying a new tool, look at the documentation and see some examples (videos for example work great with me).

try without using the same instance.
write different tests without clearing browser and cookies.

I'm not sure how to do that on cypress, cypress doesn't not support closing the browser window with a command.

It is...we just create separate specs for each

this issue is now happening on my local environment.
we must be missing a variable/flag/configuration setting (something like "no-security).
@zeljkofilipin do you remember something like this on selenium tests?

Thanks @Jpita .

I've tried both options, the first passes on my machine but fails on CI.

With the second, I get this same error locally and I used the media API for account creation.

This comment was removed by Soham.

I ran separate specs for both and both passed successfully on local. The second failed on CI with the same error.

Same thing happened here.

Since the bot logs in as admin initially, I log out initially before then logging in as a new user.

The test passes locally but fails on CI. And still, it's just the login test that fails.

@Gbahdeyboh is no longer working on this project.

Perhaps the CI needs some cypress specific config? @Soham

I also noticed re-filling and re-submitting the form after this error occurs works. I'm not exactly sure why, but that should not be the default/expected behaviour.

It has likely something to do with Kask and/or local storage as far as what I could figure from the previous discussion on this error with selenium

this is where the message is coming from:
https://translatewiki.net/w/i.php?title=Special:Translate&showMessage=sessionfailure&group=core&language=en-gb&filter=&optional=1&action=translate

By searching mediawiki core code for that key, I found this:

	protected static $messages = [
		'authform-newtoken' => 'nocookiesfornew',
		'authform-notoken' => 'sessionfailure',
		'authform-wrongtoken' => 'sessionfailure',
	];

it seems we are missing a token to avoid this problem

Yeah, seems so.

I think Soham was able to find a way around it, by refilling the form after the error occurs. Not exactly sure.

Change 616567 had a related patch set uploaded (by AlQaholic007; owner: AlQaholic007):
[mediawiki/core@master] WIP Cypress: Fix User:should be able to login test

https://gerrit.wikimedia.org/r/616567

Change 617081 had a related patch set uploaded (by AlQaholic007; owner: AlQaholic007):
[mediawiki/core@master] WIP Cypress: Separate patches for login and create account

https://gerrit.wikimedia.org/r/617081

zeljkofilipin moved this task from In Progress to Waiting/Blocked on the User-Soham board.

Change 617081 abandoned by AlQaholic007:
[mediawiki/core@master] WIP Cypress: Separate patches for login and create account

Reason:

https://gerrit.wikimedia.org/r/617081

Change 616567 abandoned by AlQaholic007:
[mediawiki/core@master] WIP Cypress: Fix User:should be able to login test

Reason:

https://gerrit.wikimedia.org/r/616567