Page MenuHomePhabricator

Set up periodic maintenance jobs for ORES cache
Closed, DeclinedPublic

Description

Proposed job schedule:

  • CheckModelVersions should run something like hourly. This period determines how much data will be incorrectly labelled with an older model version, and therefore be unnecessarily refreshed.
  • PurgeScoreCache can run daily.

TODO:

  • There should be a "dirty" flag set and cleared on the ores_model table dependending on whether we know there is old data in the cache.
  • Consider purging cached scores on revisions older than 30 days?

Event Timeline

awight raised the priority of this task from to Medium.
awight updated the task description. (Show Details)
awight added subscribers: Legoktm, Halfak, Ladsgroup and 3 others.

Proposed job schedule:

  • CheckModelVersions should run something like hourly. This period determines how much data will be incorrectly labelled with an older model version, and therefore be unnecessarily refreshed.

Too fast. Models get updated once in a while and in that case manual run is much more desirable.

  • PurgeScoreCache can run daily.

Too fast. See above.

TODO:

  • There should be a "dirty" flag set and cleared on the ores_model table dependending on whether we know there is old data in the cache.

Schema has changed and this flag is already there.

  • Consider purging cached scores on revisions older than 30 days?

No, if score of an edit in a certain version changes, we would have much bigger problems. It's a big sign of "something is wrong"