coroutine.cc:134: void* find_thread_id_key(void*): Assertion `thread_id_key != 0x7777' failed`
Closed, ResolvedPublic
Actions

Description

Failing jobs:

selenium-daily-beta-AdvancedSearch
selenium-daily-beta-CirrusSearch
selenium-daily-beta-Minerva TypeError: currentPage.includes is not a function
selenium-daily-beta-Popups
selenium-daily-beta-TwoColConflict
selenium-daily-beta-WikibaseLexeme (various error messages)

The likely cause is 890944 (jjb: Switch incidental uses of nodejs images from 14 to 16). The fix is a partial revert in 896026.

I have updated jobs (revering node.js version from v16 to v14) and after re-running the jobs, the error messages were gone.

Details

	Subject	Repo	Branch	Lines +/-
	Revert "Docker: Drop node14 images, all now unused"	integration/config	master	+392 -0
	Partial revert "jjb: Switch incidental uses of nodejs images from 14 to 16"	integration/config	master	+1 -1

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved		zeljkofilipin	T330426 selenium-daily Jenkins jobs failing with `node: ../src/coroutine.cc:134: void* find_thread_id_key(void*): Assertion `thread_id_key != 0x7777' failed`
Resolved		None	T293072 Refactor Popups WebdriverIO tests from sync to async mode
Duplicate	BUG REPORT	zeljkofilipin	T331063 Popups browser tests failing with `08:25:51 node: ../src/coroutine.cc:134: void* find_thread_id_key(void*): Assertion `thread_id_key != 0x7777' failed.`

Event Timeline

bwang created this task.Feb 23 2023, 7:57 PM

Restricted Application added subscribers: Masumrezarock100, Aklapper. · View Herald TranscriptFeb 23 2023, 7:57 PM

bwang renamed this task from Minerva selenium tests are failing to Page preview and Minerva selenium tests failing.Feb 23 2023, 8:00 PM

bwang added a project: Page-Previews.

bwang updated the task description. (Show Details)

ovasileva triaged this task as Medium priority.Mar 6 2023, 10:52 PM

ovasileva moved this task from Incoming to Current Quarter on the Web-Team-Backlog board.

Jdlrobson renamed this task from Page preview and Minerva selenium tests failing to Page preview and Minerva selenium tests failing (node: ../src/coroutine.cc:134: void* find_thread_id_key(void*): Assertion `thread_id_key != 0x7777' failed).Mar 8 2023, 6:44 PM

Jdlrobson merged a task: T331063: Popups browser tests failing with `08:25:51 node: ../src/coroutine.cc:134: void* find_thread_id_key(void*): Assertion `thread_id_key != 0x7777' failed.`.

Jdlrobson added projects: Quality-and-Test-Engineering-Team, ci-test-error, User-zeljkofilipin.

Jdlrobson added subscribers: Jdlrobson, zeljkofilipin, Jdforrester-WMF.

Jdforrester-WMF added a subtask: T293072: Refactor Popups WebdriverIO tests from sync to async mode.Mar 8 2023, 8:00 PM

zeljkofilipin changed the task status from Open to In Progress.Mar 9 2023, 8:36 AM

zeljkofilipin claimed this task.

zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.

This is probably caused by updating nodejs from v14 to v16. As far as I can see, this is the job configuration change that does it.

https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-daily-beta-Popups/jobConfigHistory/showDiffFiles?timestamp1=2022-12-08_16-07-05&timestamp2=2023-02-22_03-06-29

Change 896026 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[integration/config@master] WIP Revert "jjb: Switch incidental uses of nodejs images from 14 to 16"

https://gerrit.wikimedia.org/r/896026

gerritbot added a project: Patch-For-Review.Mar 9 2023, 8:47 AM

I've reverted selenium-daily-beta-Popups back to node14 and it's not longer failing (diff).

zeljkofilipin renamed this task from Page preview and Minerva selenium tests failing (node: ../src/coroutine.cc:134: void* find_thread_id_key(void*): Assertion `thread_id_key != 0x7777' failed) to selenium-daily Jenkins jobs failing with `node: ../src/coroutine.cc:134: void* find_thread_id_key(void*): Assertion `thread_id_key != 0x7777' failed`.Mar 9 2023, 10:49 AM

zeljkofilipin raised the priority of this task from Medium to High.

zeljkofilipin added projects: Browser-Tests, MediaWiki-Core-Tests.

zeljkofilipin moved this task from Inbox to Selenium on the MediaWiki-Core-Tests board.

zeljkofilipin updated the task description. (Show Details)Mar 9 2023, 11:02 AM

zeljkofilipin updated the task description. (Show Details)Mar 9 2023, 11:06 AM

zeljkofilipin updated the task description. (Show Details)Mar 9 2023, 11:17 AM

zeljkofilipin updated the task description. (Show Details)

selenium-daily-beta-WikibaseLexeme (various error messages)

The WikibaseLexeme daily browser tests have been flaky for a while, I’m afraid (partly due to beta cluster instability, IIRC) – unless you’ve seen the assertion error message there, I don’t think it’s related to this task. (I don’t see that message in the latest two failing builds.)

This is because we use wdio-sync which depends on fibers https://www.npmjs.com/package/fibers-nodejs which is no more supported https://github.com/laverdet/node-fibers#readme

Update [April 13th, 2021]
Fibers is not compatible with nodejs v16.0.0 or later. Unfortunately, v8 commit dacc2fee0f is a breaking change and workarounds are non-trivial.

We need to get rid of wdio-sync which is T256626: Refactor WebdriverIO tests from sync to async mode and is a blocker for the NodeJS 16 upgrade T314470.

As far as CI is concerned, for recent branches we will switch to NodeJS 16 as soon as wdio-sync got phased out. For the old branch I think we can stick the job to NodeJS 14 since I don't think we can easily backport the removal of wdio-sync.

Mentioned in SAL (#wikimedia-releng) [2023-03-09T11:36:59Z] <hashar> Rolling back selenium-daily Jenkins jobs back to NodeJS 14 | https://gerrit.wikimedia.org/r/c/integration/config/+/896026/ | T330426