Page MenuHomePhabricator
Create Task
Maniphest T358058

Custom Type Converters Are Not Working
Closed, ResolvedPublicBUG REPORT
Actions

Description

Description

Steps to reproduce (step by step instructions, with links, commands and necessary data to reproduce the error)

  1. define a type with custom type converters (Z4K8 and Z4K7)
  2. reference those custom type converters by their ZIDs in the type definition
  3. attempt to run a native-code function using that type

Observed behavior

  • blows up

Expected behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

  • would not blow up

Completion checklist

Details

SubjectRepoBranchLines +/-
wikifunctions: Upgrade orchestrator from 2024-02-12-155846 to 2024-02-22-165335operations/deployment-chartsmaster+1 -1
wikifunctions: Upgrade orchestrator from 2024-02-12-155846 to 2024-02-26-150614operations/deployment-chartsmaster+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

cmassaro created this task.Feb 20 2024, 10:03 PM
CodeReviewBot added a comment.Feb 20 2024, 10:03 PM

apine updated https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/136

Fully resolve type converter objects.

ecarg subscribed.EditedFeb 21 2024, 4:57 AM

lol the description on behaviors love it XD

edit: I took a look at your fixes in the MR, but would you be able to provide a TLDR on how this would blow up?

Jdforrester-WMF changed the task status from Open to In Progress.Feb 21 2024, 4:28 PM
Jdforrester-WMF assigned this task to cmassaro.
Jdforrester-WMF added a parent task: T297509: User-Supplied Native Code Serialization/Deserialization for ZObjects.
CodeReviewBot added a comment.Feb 22 2024, 4:53 PM

jforrester merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/136

Fully resolve type converter objects.

cmassaro added a comment.Feb 22 2024, 5:01 PM

@ecarg Yes, sure! It ends up looking like a non-specific error in the executor. But the reason is as follows:

  1. Avro serialization code is called in function-orchestrator:src/Evaluator.js; it doesn't see the Z16K1 or Z16K2 for the type converters, so it helpfully leaves those blank;
  2. what is sent to the evaluator, then, does not have the serializer and deserializer fields populated, EVEN THOUGH the code implementation expects type conversion to have taken place;
  3. some piece of code that expects to see a custom type instead sees a ZObject, and mayhem ensues. For example, if an Integer type is supposed to have been converted to a Python int, the consuming code will instead receive a dict (well, really, a thin wrapper around a dict). It will treat that object as an int, so something bad will eventually happen.
Jdforrester-WMF moved this task from To triage to To deploy on the Abstract Wikipedia team board.Feb 22 2024, 5:37 PM
gerritbot added a comment.Feb 22 2024, 9:35 PM

Change 1005843 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2024-02-12-155846 to 2024-02-22-165335

https://gerrit.wikimedia.org/r/1005843

gerritbot added a project: Patch-For-Review.Feb 22 2024, 9:35 PM
Jdforrester-WMF triaged this task as High priority.Feb 22 2024, 10:39 PM
gerritbot added a comment.Feb 28 2024, 3:12 PM

Change 1007351 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2024-02-12-155846 to 2024-02-26-150614

https://gerrit.wikimedia.org/r/1007351

gerritbot added a comment.Feb 28 2024, 3:13 PM

Change 1007351 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2024-02-12-155846 to 2024-02-26-150614

https://gerrit.wikimedia.org/r/1007351

Jdforrester-WMF moved this task from To deploy to Verify in production on the Abstract Wikipedia team board.Feb 28 2024, 3:20 PM
gerritbot added a comment.Feb 28 2024, 3:27 PM

Change 1005843 abandoned by Jforrester:

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2024-02-12-155846 to 2024-02-22-165335

Reason:

https://gerrit.wikimedia.org/r/1005843

Jdforrester-WMF closed this task as Resolved.Mar 5 2024, 8:28 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits