Page MenuHomePhabricator

Recursive function calls causes DoS-type issue on orchestrator
Closed, ResolvedPublicBUG REPORT

Description

List of steps to reproduce (step by step, including full links if applicable):

  • Create a function and code implementation (actual code doesn't matter, I was returning the input as an example)
  • Create a second implementation that is a composition, which uses its own function as Z14K2.
  • Navigate to the function page, ensure both of these implementations are correctly associated with the function, then save.
  • Create and add a tester (the content of the tester is irrelevant) to the function.

What happens?:

The orchestrator never returns test results. In addition, I was not able to interact with the MW site at all during the time that this issue was taking place. The entire system appeared to be locked up, including on either refresh or opening new tabs/windows to different content.

What should have happened instead?:
The system should prevent recursive dereferencing to avoid this issue.

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc:

Event Timeline

Change 713640 had a related patch set uploaded (by Lindsay Wardell; author: Lindsay Wardell):

[mediawiki/services/function-orchestrator@master] Add map of previously dereferenced ZIDs to the ReferenceResolver.

https://gerrit.wikimedia.org/r/713640

Change 713640 merged by jenkins-bot:

[mediawiki/services/function-orchestrator@master] Add map of previously dereferenced ZIDs to the ReferenceResolver.

https://gerrit.wikimedia.org/r/713640

Example ZObject:

{
    "Z1K1": "Z2",
    "Z2K1": "Z10028",
    "Z2K2": {
        "Z1K1": "Z14",
        "Z14K1": "Z10026",
        "Z14K2": {
            "Z1K1": "Z7",
            "Z7K1": "Z10026",
            "Z10026K1": {
                "Z1K1": "Z18",
                "Z18K1": {
                    "Z1K1": "Z6",
                    "Z6K1": "Z10026K1"
                }
            }
        }
    },
    "Z2K3": {
        "Z1K1": "Z12",
        "Z12K1": [
            {
                "Z1K1": "Z11",
                "Z11K1": "Z1002",
                "Z11K2": "recursive composition"
            }
        ]
    },
    "Z2K4": {
        "Z1K1": "Z32",
        "Z32K1": [
            {
                "Z1K1": "Z31",
                "Z31K1": "Z1002",
                "Z31K2": []
            }
        ]
    }
}

Change 713699 had a related patch set uploaded (by Lindsay Wardell; author: Lindsay Wardell):

[mediawiki/services/function-orchestrator@master] Select a random implementation (instead of defaulting to first composition)

https://gerrit.wikimedia.org/r/713699

Change 713699 merged by jenkins-bot:

[mediawiki/services/function-orchestrator@master] Select a random implementation (instead of defaulting to first composition)

https://gerrit.wikimedia.org/r/713699

Jdforrester-WMF assigned this task to Lindsaykwardell.
Jdforrester-WMF subscribed.

We'll call this Resolved for now (there's a lot more work on resource constraints and infinite recursion etc. to do, but that's tracked elsewhere).