Page MenuHomePhabricator

super-detect-noop: Support recognizing and updating subsets within an array
Closed, ResolvedPublic

Description

We want to store predictions from multiple models in the same elasticsearch field. The idea is that each prediction will be prefixed with the model, and when performing an update we want to remove all predictions for a provided model and replace them with the new set. We want to keep all predictions that came from models not mentioned in the update. This can be accomplished by adding another handler to the super-detect-noop script in the search-extra plugin.

Example _source of stored document:

{
    "ores_predictions": [
        "articletopic/Foo|123",
        "articletopic/Bar|984",
        "drafttopic/Foo|444"
    }
}

When an update comes in containing ["articletopic/Foo|333"] all existing articletopic predictions should be removed from the array and replaced with the new predictions. The resulting _source would be:

{
    "ores_predictions": [
        "articletopic/Foo|333",
        "drafttopic/Foo|444"
    }
}

For BC reasons there will also be unprefixed topics. Essentially there could also be a plain Foo|333 All predictions without the separator for defining the model should probably be considered an additional unnamed grouping.

Event Timeline

Change 605348 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[search/extra@master] super-detect-noop: Recognize and update subsets within an array

https://gerrit.wikimedia.org/r/605348

Change 605348 merged by jenkins-bot:
[search/extra@master] super-detect-noop: Recognize and update subsets within an array

https://gerrit.wikimedia.org/r/605348