Page MenuHomePhabricator
Create Task
Maniphest T386433

create a unit test to count recursive calls for builtin zobjects
Closed, ResolvedPublic
Actions

Description

What/Why:
While trying out this wikifunctions test case; we are seeing a blank in the (UI) results and when we observe the backend logstash, there's evidence that that this single test triggers over 500 recursive calls on execute(), notably for a builtin Z881. For builtins, the orchestrator code does not count the number of calls as it does for other zobjects so a higher number is not unexpected, however definitely not in the 500s. It would be ideal to get this number below 100.

How:

  1. Add a unit test counting a builtin function
  2. Observe what happens locally?
  3. Identify why the hundreds of calls; and perhaps solution may involve limiting the number of calls eventually.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
wikifunctions: Update orchestrator from 2025-02-20-140756 to 2025-02-25-210518operations/deployment-chartsmaster+1 -1
Customize query in gerrit
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
add execution call countrepos/abstract-wiki/wikifunctions/function-orchestrator!287ecarggrace/T386433/builtin-call-count-unit-testmain
Customize query in GitLab

Related Objects
Search...

Event Timeline

ecarg created this task.Feb 14 2025, 7:28 AM
ecarg triaged this task as High priority.
ecarg lowered the priority of this task from High to Medium.
ecarg added a project: Wikifunctions Improve performance.
ecarg moved this task from To Triage to 25Q3 (Jan–Mar) on the Abstract Wikipedia team board.
ecarg edited projects, added Abstract Wikipedia team (25Q3 (Jan–Mar)); removed Abstract Wikipedia team.
ecarg moved this task from Incoming to Ready, Quarterly Work on the Abstract Wikipedia team (25Q3 (Jan–Mar)) board.
CodeReviewBot added a project: Patch-For-Review.Feb 14 2025, 10:29 PM

ecarg opened https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/287

Draft: add call count unit test

DMartin-WMF added subscribers: cmassaro, DMartin-WMF.Feb 14 2025, 11:05 PM

Thanks, @ecarg . It wouldn't surprise me if our ZObject for a large lexeme like L1882 has over 500 items in lists. Lexemes (and each of their nested lexeme forms and senses) include lists of statements and glosses which can get lengthy. However, it's not clear to me why execute needs to iterate over these lists. I thought we were moving away from resolution and validation of ZObjects that we create from Wikidata content. @cmassaro would likely know more about this.

ecarg changed the task status from Open to In Progress.Feb 14 2025, 11:49 PM
DSantamaria changed the task status from In Progress to Open.Feb 17 2025, 3:46 PM
ecarg changed the task status from Open to In Progress.Feb 18 2025, 4:08 PM
ecarg moved this task from Ready, Quarterly Work to In Engineering on the Abstract Wikipedia team (25Q3 (Jan–Mar)) board.
ecarg moved this task from In Engineering to In Code review on the Abstract Wikipedia team (25Q3 (Jan–Mar)) board.Feb 23 2025, 7:41 AM
CodeReviewBot added a comment.Feb 25 2025, 8:56 PM

jforrester merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/287

add execution call count

Maintenance_bot removed a project: Patch-For-Review.Feb 25 2025, 9:31 PM
DSantamaria added subscribers: Jdforrester-WMF, DSantamaria.Feb 26 2025, 6:22 AM

@Jdforrester-WMF If it is merged, should it be in "Ready to Deploy"?

gerritbot added a comment.Feb 26 2025, 2:53 PM

Change #1122963 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Update orchestrator from 2025-02-20-140756 to 2025-02-25-210518

https://gerrit.wikimedia.org/r/1122963

gerritbot added a project: Patch-For-Review.Feb 26 2025, 2:53 PM
gerritbot added a comment.Feb 26 2025, 3:29 PM

Change #1122963 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Update orchestrator from 2025-02-20-140756 to 2025-02-25-210518

https://gerrit.wikimedia.org/r/1122963

Maintenance_bot removed a project: Patch-For-Review.Feb 26 2025, 3:31 PM
Jdforrester-WMF moved this task from In Code review to Needs Sign-off on the Abstract Wikipedia team (25Q3 (Jan–Mar)) board.Feb 26 2025, 3:39 PM

For sign-off by workstream lead, @ecarg.

ecarg added a comment.EditedFeb 27 2025, 11:15 PM

Investigation notes:

Most to all of the zobjects that get fed to execute() over the course of the 500+ calls in local env look like this:

<ref *1> ZWrapper {
  original_: Map(3) {
    'Z1K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    },
    'Z7K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    },
    'Z881K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    }
  },
  resolved_: Map(0) {},
  resolvedEphemeral_: Map(0) {},
  keys_: Set(3) { 'Z1K1', 'Z7K1', 'Z881K1' },
  scope_: null,
  parent_: ZWrapper {
    original_: Map(1) { 'Z1K1' => [Circular *1] },
    resolved_: Map(0) {},
    resolvedEphemeral_: Map(0) {},
    keys_: Set(1) { 'Z1K1' },
    scope_: null,
    parent_: ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: [Map],
      keys_: [Set],
      scope_: null,
      parent_: [ZWrapper],
      isValidated_: false,
      isFullyResolved_: false
    },
    isValidated_: false,
    isFullyResolved_: false
  },
  isValidated_: false,
  isFullyResolved_: false
}

well, here's a diff one:

 ZWrapper {
  original_: Map(4) {
    'Z1K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    },
    'Z7K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    },
    'Z19243K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    },
    'Z19243K2' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    }
  },
  resolved_: Map(0) {},
  resolvedEphemeral_: Map(0) {},
  keys_: Set(4) { 'Z1K1', 'Z7K1', 'Z19243K1', 'Z19243K2' },
  scope_: null,
  parent_: ZEnvelopeWrapper {
    original_: Map(2) { 'Z1K1' => [ZWrapper], 'Z22K1' => [Circular *1] },
    resolved_: Map(0) {},
    resolvedEphemeral_: Map(0) {},
    keys_: Set(2) { 'Z1K1', 'Z22K1' },
    scope_: EmptyFrame {
      lastFrame_: null,
      names_: Map(0) {},
      metadata_: Map(0) {},
      size_: 1
    },
    parent_: null,
    isValidated_: false,
    isFullyResolved_: false
  },
  isValidated_: false,
  isFullyResolved_: false
}
ecarg added a comment.Feb 27 2025, 11:57 PM

here's one (long) scenario of the keys of the zobject when execute gets called:

EXECUTE FUNCTION CALLED [Function: keys]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z7K1', 'Z881K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z7K1', 'Z881K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z6K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z7K1', 'Z881K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z6K1' ]
zobject.keys [ 'Z1K1', 'Z11K1', 'Z11K2' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z7K1', 'Z881K1' ]
zobject.keys [ 'Z1K1' ]
zobject.keys [ 'Z1K1', 'K1', 'K2' ]
zobject.keys [ 'Z1K1', 'Z12K1' ]
zobject.keys [ 'Z1K1', 'Z3K1', 'Z3K2', 'Z3K3' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z7K1', 'Z881K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z7K1', 'Z881K1' ]
zobject.keys [ 'Z1K1', 'Z6K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z7K1', 'Z881K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z6K1' ]
zobject.keys [ 'Z1K1', 'Z11K1', 'Z11K2' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z7K1', 'Z881K1' ]
zobject.keys [ 'Z1K1' ]
zobject.keys [ 'Z1K1', 'K1', 'K2' ]
zobject.keys [ 'Z1K1', 'Z12K1' ]
zobject.keys [ 'Z1K1', 'Z3K1', 'Z3K2', 'Z3K3' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z7K1', 'Z881K1' ]
zobject.keys [ 'Z1K1' ]
zobject.keys [ 'Z1K1', 'K1', 'K2' ]
zobject.keys [ 'Z1K1', 'K1', 'K2' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z4K1', 'Z4K2', 'Z4K3' ]
zobject.keys [ 'Z1K1', 'Z22K1' ]
zobject.keys [ 'Z1K1', 'Z22K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z22K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z22K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z22K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z22K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z22K1' ]
zobject.keys [ 'Z1K1', 'Z9K1' ]
zobject.keys [ 'Z1K1', 'Z22K1' ]
ecarg added a comment.EditedFeb 28 2025, 9:26 PM

It doesn't look like the issue is because we 'skip' builtins, i.e. this line in execute.js:

if ( isGenericListType( zobject ) && !resolveInternals ) {...

We barely enter this code block

The one time we do, this is the zobject:

<ref *1> ZWrapper {
  original_: Map(3) {
    'Z1K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    },
    'Z7K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    },
    'Z881K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    }
  },
  resolved_: Map(0) {},
  resolvedEphemeral_: Map(0) {},
  keys_: Set(3) { 'Z1K1', 'Z7K1', 'Z881K1' },
  scope_: Frame {
    lastFrame_: Frame {
      lastFrame_: [EmptyFrame],
      names_: Map(0) {},
      metadata_: [Map],
      size_: 2
    },
    names_: Map(0) {},
    metadata_: Map(0) {},
    size_: 3
  },
  parent_: ZWrapper {
    original_: Map(3) {
      'Z1K1' => [Circular *1],
      'K1' => [ZWrapper],
      'K2' => [ZWrapper]
    },
    resolved_: Map(0) {},
    resolvedEphemeral_: Map(0) {},
    keys_: Set(3) { 'Z1K1', 'K1', 'K2' },
    scope_: null,
    parent_: ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [ZWrapper],
      isValidated_: false,
      isFullyResolved_: false
    },
    isValidated_: false,
    isFullyResolved_: false
  },
  isValidated_: false,
  isFullyResolved_: false
}
ecarg added a comment.EditedFeb 28 2025, 10:54 PM

I think we spend the most time in ZWrapper.js, this is the stack trace when we call createInternal_():

Trace
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:33:430)
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:72:41)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:39:33)
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:72:41)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:39:33)
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:72:41)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:39:33)
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:72:41)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:39:33)
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:72:41)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:39:33)
    at ZWrapper.create (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:33:42)
    at ZWrapper.resolveInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:122:655)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ZWrapper.resolveInternalHelper_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:134:620)
    at async ZWrapper.resolveEphemeralInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:163:465)
    at async ZWrapper.resolveEphemeral (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:214:205)
    at async resolveFunctionInternals (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/execute.js:84:146)
    at async executeInternal (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/execute.js:162:53)
    at async CallCounter.execute [as callable_] (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/execute.js:214:235)
    at async returnOnFirstError (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/utils.js:85:1196)
    at async orchestrate (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/orchestrate.js:25:626)
    at async Context.<anonymous> (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/test/features/v1/mockedServicesOrchestrateTest/functionCallCountTests.js:319:3)

And stack trace when populateKeys_():

Trace
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:66:118)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:40:33)
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:74:41)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:40:33)
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:74:41)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:40:33)
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:74:41)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:40:33)
    at ZWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:74:41)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:40:33)
    at ZEnvelopeWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:74:41)
    at ZEnvelopeWrapper.populateKeys_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:260:434)
    at ZWrapper.createInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:40:33)
    at ZWrapper.create (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:33:42)
    at executeInternal (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/execute.js:159:17)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async CallCounter.execute [as callable_] (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/execute.js:214:235)
    at async ZWrapper.resolveInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:124:1108)
    at async ZWrapper.resolveInternalHelper_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:136:620)
    at async ZWrapper.resolveEphemeralInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:165:465)
    at async ZWrapper.resolveEphemeral (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:216:205)
    at async ZWrapper.resolveInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:124:1682)
    at async ZWrapper.resolveInternalHelper_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:136:620)
    at async ZWrapper.resolveEphemeralInternal_ (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:165:465)
    at async ZWrapper.resolveEphemeral (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/ZWrapper.js:216:205)
    at async resolveFunctionInternals (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/execute.js:84:489)
    at async executeInternal (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/execute.js:162:53)
    at async CallCounter.execute [as callable_] (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/execute.js:214:235)
    at async returnOnFirstError (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/utils.js:85:1196)
    at async orchestrate (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/src/orchestrate.js:25:626)
    at async Context.<anonymous> (/Users/ecarg/Development/mediawiki/core/services/function-orchestrator/test/features/v1/mockedServicesOrchestrateTest/functionCallCountTests.js:319:3)
cmassaro added a comment.Mar 3 2025, 9:52 PM
In T386433#10592463, @ecarg wrote:

It doesn't look like the issue is because we 'skip' builtins, i.e. this line in execute.js:

if ( isGenericListType( zobject ) && !resolveInternals ) {...

We barely enter this code block

The one time we do, this is the zobject:

<ref *1> ZWrapper {
  original_: Map(3) {
    'Z1K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    },
    'Z7K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    },
    'Z881K1' => ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [Circular *1],
      isValidated_: false,
      isFullyResolved_: false
    }
  },
  resolved_: Map(0) {},
  resolvedEphemeral_: Map(0) {},
  keys_: Set(3) { 'Z1K1', 'Z7K1', 'Z881K1' },
  scope_: Frame {
    lastFrame_: Frame {
      lastFrame_: [EmptyFrame],
      names_: Map(0) {},
      metadata_: [Map],
      size_: 2
    },
    names_: Map(0) {},
    metadata_: Map(0) {},
    size_: 3
  },
  parent_: ZWrapper {
    original_: Map(3) {
      'Z1K1' => [Circular *1],
      'K1' => [ZWrapper],
      'K2' => [ZWrapper]
    },
    resolved_: Map(0) {},
    resolvedEphemeral_: Map(0) {},
    keys_: Set(3) { 'Z1K1', 'K1', 'K2' },
    scope_: null,
    parent_: ZWrapper {
      original_: [Map],
      resolved_: Map(0) {},
      resolvedEphemeral_: Map(0) {},
      keys_: [Set],
      scope_: null,
      parent_: [ZWrapper],
      isValidated_: false,
      isFullyResolved_: false
    },
    isValidated_: false,
    isFullyResolved_: false
  },
  isValidated_: false,
  isFullyResolved_: false
}

I'm not understanding this one. The builtin skipping logic is all over the place (not just here).

cmassaro added a comment.Mar 3 2025, 9:53 PM

It does look like we're resolving a ton of Z881s, and it's very expensive to do so. I wonder if that check you pointed out is just buggy. In any case, I think we can do much more shortcutting around Z881 as a first pass.

ecarg added a comment.Mar 3 2025, 10:55 PM

More notes:

I'm wondering if object reinstantation has something to do with redundant calls; places like
let result;
		if ( isZEnvelope( zobjectJSON ) ) {
			result = new ZEnvelopeWrapper(); // eslint-disable-line no-use-before-define
		} else {
			result = new ZWrapper();
		}
		result.setScope( scope );
		result.setIsValidated( isValidated );
		result.setIsFullyResolved( isFullyResolved );
		result.parent_ = parentPointer; // will use parent in case scope does not exist
		result.populateKeys_( zobjectJSON, isValidated, isFullyResolved );
return result;
and/or
let result = await executeInternal(
		zobject, invariants, doValidate,
		implementationSelector, resolveInternals, topLevel, eagerlyEvaluateWithCallCount );
	if ( topLevel ) {...
...
return result;
ecarg added a comment.EditedMar 3 2025, 10:55 PM

it also seems like scope is not being passed as expected. It gets initialized correctly at ZWrapper.create():

 scope:  EmptyFrame {
  lastFrame_: null,
  names_: Map(0) {},
  metadata_: Map(0) {},
  size_: 1
}

but then if I check scope in the subsequent functions, the value is null throughout until another ZWrapper.create()

It seems like it is null anytime the state is not passed from function makeWrappedResultEnvelope( ...args )
and then from there, in all other functions in ZWrapper class following create() and createInternal() :

  1. function makeWrappedResultEnvelope( ...args )
  2. ZWrapper.create() => scope != null, size 1
  3. .createInternal() => scope != null, size: 1

all other subsequent functions => scope = null

Scope is null whenever parentPointer is not null. Those two should be mutually exclusive. So it's perfectly normal for scope to be null.

  • observe if places where both scope and parentPointer are null
ecarg added a comment.Mar 5 2025, 4:30 PM

scope size remains same and doesn't update correctly

ecarg added a comment.Mar 5 2025, 11:21 PM

reduce duplicate calls during ZWrapper: https://phabricator.wikimedia.org/T388075

ecarg closed this task as Resolved.EditedMar 6 2025, 5:37 AM

objective of the ticket resolved; but using this as a notepad to document relevant findings

ecarg added a parent task: T383157: [25Q3] Improve performance and drive down tech debt & developer inhibitors.Mar 14 2025, 10:30 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits