Summary
The IP Info selenium tests are flakily timing out. In the latest instance I saw they hung for 20 minutes.
Background
- The IP Info selenium tests check that the JS frontend works, including setting the IPInfo use agreement
- Selenium tests are often more flaky than other testing methods, but it is the only option we have for e2e testing in a browser
- If they are flaky they often can be self-fixed by CI re-running the tests, but this can fail if they fail multiple times
- Because the test is flaky it suggests that the problem isn't caused by a problem with the code being tested but instead in how the tests are written and/or execute
- However, we cannot rule out that there is some race condition caused by the code we are testing
Technical notes
The failing gate-and-submit CI build was https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php81/19822/console and the extract is:
20:50:21 INFO:quibble.commands:Running webdriver test in /workspace/src/extensions/IPInfo 20:50:24 npm warn deprecated @types/easy-table@1.2.0: This is a stub types definition. easy-table provides its own type definitions, so you do not need this installed. 20:50:24 npm warn deprecated har-validator@5.1.5: this library is no longer supported 20:50:24 npm warn deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported 20:50:24 npm warn deprecated glob@8.1.0: Glob versions prior to v9 are no longer supported 20:50:24 npm warn deprecated glob@8.1.0: Glob versions prior to v9 are no longer supported 20:50:24 npm warn deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. 20:50:24 npm warn deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142 20:50:27 20:50:27 added 797 packages, and audited 798 packages in 6s 20:50:27 20:50:27 133 packages are looking for funding 20:50:27 run `npm fund` for details 20:50:27 20:50:27 16 vulnerabilities (4 moderate, 12 high) 20:50:27 20:50:27 To address issues that do not require attention, run: 20:50:27 npm audit fix 20:50:27 20:50:27 To address all issues possible (including breaking changes), run: 20:50:27 npm audit fix --force 20:50:27 20:50:27 Some issues need review, and may require choosing 20:50:27 a different dependency. 20:50:27 20:50:27 Run `npm audit` for details. 20:50:27 20:50:27 > selenium-test 20:50:27 > wdio tests/selenium/wdio.conf.js 20:50:27 20:50:28 20:50:28 Execution of 2 workers started at 2025-06-03T19:50:28.332Z 20:50:28 20:50:28 Setting up modified /workspace/src/LocalSettings.php 20:50:28 Restarting php8.1-fpm 20:50:30 [0-0] RUNNING in chrome - /tests/selenium/specs/contributions.js 20:50:30 [0-1] RUNNING in chrome - /tests/selenium/specs/ipcontributions.js 20:50:51 [0-1] PASSED in chrome - /tests/selenium/specs/ipcontributions.js 20:51:51 [0-0] Error in "IPInfo on Special:Contributions.should show geo data for IP with edits after accepting agreement" 20:51:51 Error: Timeout of 60000ms exceeded. The execution in the test "IPInfo on Special:Contributions should show geo data for IP with edits after accepting agreement" took too long. Try to reduce the run time or increase your timeout for test specs (https://webdriver.io/docs/timeouts). (/workspace/src/extensions/IPInfo/tests/selenium/specs/contributions.js) 20:51:51 at createTimeoutError (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/errors.js:498:15) 20:51:51 at Runnable._timeoutError (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/runnable.js:429:10) 20:51:51 at Timeout.<anonymous> (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/runnable.js:244:24) 20:51:51 at listOnTimeout (node:internal/timers:581:17) 20:51:51 at process.processTimers (node:internal/timers:519:7) 20:52:51 [0-0] Error in "IPInfo on Special:Contributions.should show geo data for IP with edits if agreement was already accepted" 20:52:51 Error: Timeout of 60000ms exceeded. The execution in the test "IPInfo on Special:Contributions should show geo data for IP with edits if agreement was already accepted" took too long. Try to reduce the run time or increase your timeout for test specs (https://webdriver.io/docs/timeouts). (/workspace/src/extensions/IPInfo/tests/selenium/specs/contributions.js) 20:52:51 at createTimeoutError (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/errors.js:498:15) 20:52:51 at Runnable._timeoutError (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/runnable.js:429:10) 20:52:51 at Timeout.<anonymous> (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/runnable.js:244:24) 20:52:51 at listOnTimeout (node:internal/timers:581:17) 20:52:51 at process.processTimers (node:internal/timers:519:7) 20:53:51 [0-0] Error in "IPInfo on Special:Contributions.should show geo data for temp user with edits after accepting agreement" 20:53:51 Error: Timeout of 60000ms exceeded. The execution in the test "IPInfo on Special:Contributions should show geo data for temp user with edits after accepting agreement" took too long. Try to reduce the run time or increase your timeout for test specs (https://webdriver.io/docs/timeouts). (/workspace/src/extensions/IPInfo/tests/selenium/specs/contributions.js) 20:53:51 at createTimeoutError (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/errors.js:498:15) 20:53:51 at Runnable._timeoutError (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/runnable.js:429:10) 20:53:51 at Timeout.<anonymous> (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/runnable.js:244:24) 20:53:51 at listOnTimeout (node:internal/timers:581:17) 20:53:51 at process.processTimers (node:internal/timers:519:7) 20:54:51 [0-0] Error in "IPInfo on Special:Contributions.should show geo data for temp user with edits if agreement was already accepted" 20:54:51 Error: Timeout of 60000ms exceeded. The execution in the test "IPInfo on Special:Contributions should show geo data for temp user with edits if agreement was already accepted" took too long. Try to reduce the run time or increase your timeout for test specs (https://webdriver.io/docs/timeouts). (/workspace/src/extensions/IPInfo/tests/selenium/specs/contributions.js) 20:54:51 at createTimeoutError (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/errors.js:498:15) 20:54:51 at Runnable._timeoutError (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/runnable.js:429:10) 20:54:51 at Timeout.<anonymous> (/workspace/src/extensions/IPInfo/node_modules/mocha/lib/runnable.js:244:24) 20:54:51 at listOnTimeout (node:internal/timers:581:17) 20:54:51 at process.processTimers (node:internal/timers:519:7) 21:13:17 Build was aborted 21:13:17 Aborted by Dreamy Jazz
Acceptance criteria
- The selenium tests are dropped, along with removal of any supporting code