Page MenuHomePhabricator
Create Task
Maniphest T392905

invalid result validation in one test causes indefinite "running" in all others
Closed, ResolvedPublicBUG REPORT
Actions

Description

Steps to replicate the issue (include links if applicable):

Similar behavior observed

What happens?:

image.png (817×556 px, 39 KB)

  • The error created as a response is so big that the API fails with "This result was truncated because it would otherwise be larger than the limit of 12,582,912 bytes." (see api call)

What should have happened instead?:

  • It's understandable that the broken test itself fails or exhibits strange behaviour. But this should not affect the outcome of all the others, especially if it's not connected.
  • UI should replace "Running" with "Failed" after the response is received

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Build ApiError when response returns 200 but contains warningsmediawiki/extensions/WikiLambdamaster+62 -1
Customize query in gerrit

Related Objects

Event Timeline

99of9 created this task.Apr 29 2025, 1:34 PM
gengh updated the task description. (Show Details)Apr 29 2025, 2:02 PM
Jdforrester-WMF subscribed.Apr 29 2025, 4:31 PM

There's a JS fatal in the console:

Uncaught (in promise) TypeError: error.messageOrFallback is not a function

at testResults.js:274:26

So we should definitely handle that, alongside the long response.

99of9 added a comment.Apr 30 2025, 1:40 AM

Similar behaviour at the built-in function https://www.wikifunctions.org/view/en/Z811

99of9 added a comment.Apr 30 2025, 1:10 PM

Also at https://www.wikifunctions.org/view/en/Z12864

gengh subscribed.Apr 30 2025, 5:16 PM

For all these objects, the evaluator is generating an error that is too big to return as a response.

  • Z507K1 contains the whole dereferenced object (e.g. 46000 lines of JSON) making it too big for a response and also unreadable
  • Z507K2 contains the following error:
Function evaluation failed with status 500: 

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>TypeError: Cannot read properties of undefined (reading '0')
    at WikimediaEcsFormatMiddleware.transform (/srv/service/node_modules/@wikimedia/service-utils/dist/index.js:29:39)
    at Format.transform (/srv/service/node_modules/logform/combine.js:20:24)
    at DerivedLogger._transform (/srv/service/node_modules/winston/lib/winston/logger.js:314:29)
    at Transform._read (/srv/service/node_modules/readable-stream/lib/_stream_transform.js:166:10)
    at Transform._write (/srv/service/node_modules/readable-stream/lib/_stream_transform.js:155:83)
    at doWrite (/srv/service/node_modules/readable-stream/lib/_stream_writable.js:390:139)
    at writeOrBuffer (/srv/service/node_modules/readable-stream/lib/_stream_writable.js:381:5)
    at Writable.write (/srv/service/node_modules/readable-stream/lib/_stream_writable.js:302:11)
    at DerivedLogger.value (/srv/service/node_modules/winston/lib/winston/logger.js:67:18)
    at DerivedLogger.log (/srv/service/node_modules/winston/lib/winston/logger.js:225:14)
</pre>
</body>
</html>

Things to fix:

  • When creating error Z507, set Z507K1 with the initial function call, instead of the same in its current dereferenced shape. This not only would help with readability but would remove the issue of having responses be too large.
  • Handle errors in the front-end perform test request, so that a response of this kind renders "Failed" instead of remaining in "Running..."
  • Find out and fix the origin of the error
gengh added a comment.Apr 30 2025, 5:19 PM

Seems to be related to T391435: Investigate performance of Python implementations where there is an argument containing an explicitly typed list

gengh claimed this task.Apr 30 2025, 5:20 PM
ecarg mentioned this in T393130: handle large log response.May 1 2025, 9:33 PM
99of9 added a comment.May 2 2025, 1:13 PM

Also at https://www.wikifunctions.org/view/en/Z24185

gengh mentioned this in T391435: Investigate performance of Python implementations where there is an argument containing an explicitly typed list.May 7 2025, 12:57 PM

The different underlying issues are being addressed in different phab tasks:
T393130: handle large log response: some fixes merged, but more to come
T392905: invalid result validation in one test causes indefinite "running" in all others: fixed and merged
T393152: Service utils logger fails with message containing stack trace: service-utils related issue, related fix is merged

Jdforrester-WMF triaged this task as High priority.May 7 2025, 3:39 PM
Jdforrester-WMF edited projects, added: Abstract Wikipedia team (25Q4 (Apr–Jun)), Essential-Work; removed: Abstract Wikipedia team.
Jdforrester-WMF moved this task from Incoming to In Engineering on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.
gerritbot added a comment.May 8 2025, 9:13 PM

Change #1143674 had a related patch set uploaded (by Genoveva Galarza; author: Genoveva Galarza):

[mediawiki/extensions/WikiLambda@master] Build ApiError when response returns 200 but contains warnings

https://gerrit.wikimedia.org/r/1143674

gerritbot added a project: Patch-For-Review.May 8 2025, 9:13 PM
gengh added a comment.May 8 2025, 9:23 PM

If response contains warnings and no data, we are ending the request with "Failure" and displaying the reason in the error message:

Screenshot from 2025-05-08 23-07-56.png (784×521 px, 48 KB)

gerritbot added a comment.May 8 2025, 10:05 PM

Change #1143674 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Build ApiError when response returns 200 but contains warnings

https://gerrit.wikimedia.org/r/1143674

Maintenance_bot removed a project: Patch-For-Review.May 8 2025, 10:30 PM
ReleaseTaggerBot added a project: MW-1.45-notes (1.45.0-wmf.1; 2025-05-13).May 8 2025, 11:00 PM
DSantamaria changed the task status from Open to In Progress.May 10 2025, 10:09 AM
gengh moved this task from In Engineering to In Code review on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.May 12 2025, 8:05 AM
Jdforrester-WMF moved this task from In Code review to Ready to deploy on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.May 14 2025, 3:41 PM
gengh moved this task from Ready to deploy to Needs Sign-off on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.May 19 2025, 8:15 AM
gengh closed this task as Resolved.May 19 2025, 8:17 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits