Page MenuHomePhabricator
Create Task
Maniphest T390550

Implement Object and Wikidata entity batching mechanisms in the function-orchestrator to drive up user experienced performance when making function calls and reduce load on production wikis
Closed, ResolvedPublic
Actions

Description

Description

After setting up our caching (by way of the available memcache or custom side car), we want to set up the ability to batch requests in order to relieve the latency during over network requests.

Acceptance Criteria

  • A function call's local runtime has been reduced to ~1615ms

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
wikifunctions: Enable memcached-based batching for ZObjectsoperations/deployment-chartsmaster+1 -0
wikifunctions: Update orchestrator from 2025-06-10-144243 to 2025-06-18-130945operations/deployment-chartsmaster+1 -1
Customize query in gerrit
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Enable memcached-based batching for ZObjectsrepos/abstract-wiki/wikifunctions/function-orchestrator!384apineapine-batchitupmain
Customize query in GitLab

Related Objects
Search...

Event Timeline

Jdforrester-WMF created this task.Mar 31 2025, 1:45 PM
Jdforrester-WMF removed a project: Epic.
DSantamaria assigned this task to ecarg.Apr 3 2025, 2:54 PM
DSantamaria added a project: Workstreams.Apr 7 2025, 12:55 PM
ecarg updated the task description. (Show Details)Apr 8 2025, 1:33 AM
DSantamaria changed the status of subtask T391308: decide on batching mechanism from Open to In Progress.Apr 16 2025, 4:21 PM
Jdforrester-WMF moved this task from Incoming to Ready on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.Apr 16 2025, 8:19 PM
Jdforrester-WMF mentioned this in T388075: temporarily store visited objects during ZWrapper formation.Apr 16 2025, 9:25 PM
Jdforrester-WMF added a subtask: T388075: temporarily store visited objects during ZWrapper formation.
Jdforrester-WMF triaged this task as Medium priority.Apr 16 2025, 9:33 PM
ecarg changed the status of subtask T388075: temporarily store visited objects during ZWrapper formation from In Progress to Stalled.Apr 28 2025, 11:42 PM
Jdforrester-WMF changed the task status from Open to In Progress.May 7 2025, 3:54 PM
Jdforrester-WMF moved this task from Ready to In Engineering on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.
ecarg changed the status of subtask T391308: decide on batching mechanism from In Progress to Stalled.May 9 2025, 11:25 PM
ecarg changed the status of subtask T391308: decide on batching mechanism from Stalled to In Progress.May 10 2025, 12:48 AM
DSantamaria changed the task status from In Progress to Open.May 15 2025, 5:46 PM
DSantamaria moved this task from In Engineering to Ready on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.
DSantamaria changed the status of subtask T391308: decide on batching mechanism from In Progress to Open.
DSantamaria changed the status of subtask T388075: temporarily store visited objects during ZWrapper formation from Stalled to Open.May 15 2025, 5:49 PM
cmassaro added a subtask: T394672: Spike: Optimize Further the ZID Batching Algorithms.May 19 2025, 1:46 PM
DSantamaria removed a subtask: T388075: temporarily store visited objects during ZWrapper formation.Jun 11 2025, 4:33 PM
CodeReviewBot added a project: Patch-For-Review.Jun 12 2025, 6:31 PM

apine opened https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/384

Enable memcached-based batching in the orchestrator.

Jdforrester-WMF changed the status of subtask T391308: decide on batching mechanism from Open to In Progress.Jun 13 2025, 3:00 PM
Jdforrester-WMF changed the task status from Open to In Progress.Jun 13 2025, 3:04 PM
Jdforrester-WMF moved this task from Ready to In Code review on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.
CodeReviewBot added a comment.Jun 16 2025, 11:02 PM

ecarg merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/384

Enable memcached-based batching for ZObjects

Maintenance_bot removed a project: Patch-For-Review.Jun 16 2025, 11:30 PM
Jdforrester-WMF moved this task from In Code review to Ready to deploy on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.Jun 17 2025, 2:15 PM
DSantamaria removed a subtask: T394672: Spike: Optimize Further the ZID Batching Algorithms.Jun 17 2025, 3:39 PM
gerritbot added a comment.Jun 18 2025, 12:41 PM

Change #1160760 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Update orchestrator from 2025-06-10-144243 to 2025-06-17-204731

https://gerrit.wikimedia.org/r/1160760

gerritbot added a project: Patch-For-Review.Jun 18 2025, 12:41 PM

Change #1160761 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Enable memcached-based batching for ZObjects

https://gerrit.wikimedia.org/r/1160761

gerritbot added a comment.Jun 18 2025, 2:12 PM

Change #1160760 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Update orchestrator from 2025-06-10-144243 to 2025-06-18-130945

https://gerrit.wikimedia.org/r/1160760

ecarg closed subtask T391308: decide on batching mechanism as Resolved.Jun 18 2025, 5:38 PM
gerritbot added a comment.Jun 23 2025, 7:41 PM

Change #1160761 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Enable memcached-based batching for ZObjects

https://gerrit.wikimedia.org/r/1160761

Maintenance_bot removed a project: Patch-For-Review.Jun 23 2025, 8:30 PM
Jdforrester-WMF moved this task from Ready to deploy to Needs Sign-off on the Abstract Wikipedia team (25Q4 (Apr–Jun)) board.Jul 8 2025, 4:36 PM
Jdforrester-WMF closed this task as Resolved.Jul 22 2025, 1:19 PM

Let's declare this Resolved.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits