Another reproduction on CI using check experimental this morning https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2770/console

Failure image @ https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2770/artifact/log/can-create-lexeme-with-language-item-not-bearing-language-code-statement-and-en-lemma-language.png

Again with:

09:22:27 1) NewLexeme:Page for different lemma languages can create lexeme with language item not bearing language code statement and en lemma language:
09:22:27 Cannot read property 'value' of undefined
09:22:27 running chrome
09:22:27 TypeError: Cannot read property 'value' of undefined
09:22:27     at LexemeApi.get.then (/workspace/src/extensions/WikibaseLexeme/tests/selenium/specs/new.js:152:63)

I added some debug logging to SpecialNewLexeme https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseLexeme/+/450970/1/src/Specials/SpecialNewLexeme.php

While doing "check experimental" the failure happened in the same way, with ady language code being used: https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2771/artifact/log/can-create-lexeme-with-language-item-not-bearing-language-code-statement-and-mis-lemma-language.png

In the log for the web requests (https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2771/artifact/log/mw-debug-www.log) I found:

Wikibase\Lexeme\Specials\SpecialNewLexeme::createEntityFromFormData: form data: {"lemma":"lemma-0.5735099835768849-I\u00f1t\u00ebrn\u00e2ti\u00f4n\u00e0liz\u00e6ti\u00f8n","lemma-language":"ady","lexeme-language":"Q32","lexicalcategory":"Q33"}
Wikibase\Lexeme\Specials\SpecialNewLexeme::createEntityFromFormData: entity: O:32:"Wikibase\Lexeme\DataModel\Lexeme":9:{s:36:" Wikibase\Lexeme\DataModel\Lexeme id";N;s:44:" Wikibase\Lexeme\DataModel\Lexeme statements";O:42:"Wikibase\DataModel\Statement\StatementList":1:{s:54:" Wikibase\DataModel\Statement\StatementList statements";a:0:{}}s:40:" Wikibase\Lexeme\DataModel\Lexeme lemmas";O:32:"Wikibase\DataModel\Term\TermList":1:{s:39:" Wikibase\DataModel\Term\TermList terms";a:1:{s:3:"ady";O:28:"Wikibase\DataModel\Term\Term":2:{s:42:" Wikibase\DataModel\Term\Term languageCode";s:3:"ady";s:34:" Wikibase\DataModel\Term\Term text";s:52:"lemma-0.5735099835768849-Iñtërnâtiônà lizætiøn";}}}s:49:" Wikibase\Lexeme\DataModel\Lexeme lexicalCategory";C:32:"Wikibase\DataModel\Entity\ItemId":3:{Q33}s:42:" Wikibase\Lexeme\DataModel\Lexeme language";C:32:"Wikibase\DataModel\Entity\ItemId":3:{Q32}s:39:" Wikibase\Lexeme\DataModel\Lexeme forms";O:33:"Wikibase\Lexeme\DataModel\FormSet":1:{s:40:" Wikibase\Lexeme\DataModel\FormSet forms";a:0:{}}s:40:" Wikibase\Lexeme\DataModel\Lexeme senses";O:34:"Wikibase\Lexeme\DataModel\SenseSet":1:{s:42:" Wikibase\Lexeme\DataModel\SenseSet senses";a:0:{}}s:44:" Wikibase\Lexeme\DataModel\Lexeme nextFormId";i:1;s:45:" Wikibase\Lexeme\DataModel\Lexeme nextSenseId";i:1;}

Showing that createEntityFromFormData gets given "ady" as the language code..

This data comes straight from the special page form submission which makes me think that the selenium test is for some reason not able to select en? or something else is going wrong resulting in ady being selected...

Using my debugging patch at https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseLexeme/+/450970/ I managed to get some screenshots just before the submit button is clicked on the special page.

https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2778/

The screenshot shows that en is correctly entered into the input field before the submit button is clicked....

The state of the screenshot also looks identical to the current node only job that has been passing for some time...

https://integration.wikimedia.org/ci/job/mwext-mw-selenium-node-composer-jessie/1350/
https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2778/artifact/log/can-create-lexeme-with-language-item-not-bearing-language-code-statement-and-en-lemma-language.png

The latest "check experimental" run worked.....

https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2779/console

09:04:23 [chrome #0-13] Session ID: 46b8e77e5f75012875c7013fd6a48be5
09:04:23 [chrome #0-13] Spec: /workspace/src/extensions/WikibaseLexeme/tests/selenium/specs/new.js
09:04:23 [chrome #0-13] Running: chrome
09:04:23 [chrome #0-13]
09:04:23 [chrome #0-13]   NewLexeme:Page
09:04:23 [chrome #0-13]
09:04:23 [chrome #0-13]   NewLexeme:Page
09:04:23 [chrome #0-13]       ✓ request with "createpage" right shows form
09:04:23 [chrome #0-13]
09:04:23 [chrome #0-13]   with default LexemeLanguageCodePropertyId configured
09:04:23 [chrome #0-13]       - can create lexeme with language item bearing language code statement
09:04:23 [chrome #0-13]
09:04:23 [chrome #0-13]   for different lemma languages
09:04:23 [chrome #0-13]       ✓ can create lexeme with language item not bearing language code statement and en lemma language
09:04:23 [chrome #0-13]       ✓ can create lexeme with language item not bearing language code statement and mis lemma language
09:04:23 [chrome #0-13]
09:04:23 [chrome #0-13]
09:04:23 [chrome #0-13] 3 passing (12s)
09:04:23 [chrome #0-13] 1 pending
09:04:23 [chrome #0-13]

Change 451275 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/WikibaseLexeme@master] selenium, new.js, fix assertion param order

https://gerrit.wikimedia.org/r/451275

Change 451275 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] selenium, new.js, fix assertion param order

https://gerrit.wikimedia.org/r/451275

With the patches merged we now have a slightly more informative error:

https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2785/console

11:12:33 1) NewLexeme:Page for different lemma languages can create lexeme with language item not bearing language code statement and en lemma language:
11:12:33 '{"ady":{"language":"ady","value":"lemma-0.49215704269085436-Iñtërnâtiônàlizætiøn"}}' == '{"en":{"language":"en","value":"lemma-0.49215704269085436-Iñtërnâtiônàlizætiøn"}}'
11:12:33 running chrome
11:12:33 AssertionError: '{"ady":{"language":"ady","value":"lemma-0.49215704269085436-Iñtërnâtiônàlizætiøn"}}' == '{"en":{"language":"en","value":"lemma-0.49215704269085436-Iñtërnâtiônàlizætiøn"}}'
11:12:33     at LexemeApi.get.then (/workspace/src/extensions/WikibaseLexeme/tests/selenium/specs/new.js:152:15)

I'm starting to think this is an issue with the node js selenium stuff, a video recording would be super helpful T179188

In PS5 of https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseLexeme/+/450970/ I added a bunch of screenshots to capture what happens....
They can be found @ https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/2788/ but as that will disappear I'll copy them into this comment..

Before entering the language code the form looks as expected:

"en" is then entered into the input field, and some animation / value selection can be seen happening in the following screenshot.

Note that en / English is not yet selected.

The next screenshot is taken just after "Enter" is hit by the test and now shows the "ady" language code:

After typing "en" we actually want to wait until the following state (taken from test wikidata)

Where en / English is selected from the drop down list, or alternatively click on the entry in the list that we want, once it is available.

@hoo also just pointed out that if you enter "en" and wait for the dropdown selection to select English, but you happen to have your mouse over another value in the drop down list when you hit "Enter" rather than en / English be selected, the value under your mouse will actually be selected and put in the input box.

It could be this rather than some sort of timing / animation issue, however I imagine if it were down to mouse position this would be slightly more reproducible / slightly less random.

We could one or a mix of the below:

• Waiting for the correct element to be selected in the dropdown before hitting "Enter", could also validate the selection after hitting enter?
• Make sure the mouse is nowhere near the dropdown element
• Instead of hitting Enter, just move to the next element, this should correctly leave "en" in the input

Change 451327 had a related patch set uploaded (by Pablo Grass (WMDE); owner: Pablo Grass (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] NewLexemePage: lemma language in selenium test

https://gerrit.wikimedia.org/r/451327

Unfortunately I could not come up with a completely different, more robust, approach of setting the value on the ComboBoxWidget.
I tried a couple of different ways of improving what was, making sure it is not the sheer speed at which the UI interactions are performed (the latest one going as far as not being actually fit for production). However waiting still does not seem to be enough which could point towards the mouse theory mentioned by Adam.

Will proceed tomorrow.

Tried and traded the suggestion confirmation via enter for a click on the option. Seems to improve situation, but now other tests break (T201598; likely unrelated but red no less).

Change 451832 had a related patch set uploaded (by Pablo Grass (WMDE); owner: Pablo Grass (WMDE)):
[mediawiki/extensions/Wikibase@master] ComponentInteraction: set value on combobox

https://gerrit.wikimedia.org/r/451832

Change 451832 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] ComponentInteraction: set value on combobox

https://gerrit.wikimedia.org/r/451832

Change 451327 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] NewLexemePage: lemma language in selenium test

https://gerrit.wikimedia.org/r/451327