Page MenuHomePhabricator

[backend][jobs] Improve tool insertion workflow
Closed, DuplicatePublic

Description

check_for_entry() checks if a tool is already in our db, and does nothing if it is. If it's not, it inserts it into the tools table, unless the tool is deprecated. However, a tool could already be in our db, but with outdated info. The current workflow doesn't update existing tool records.

Suggestion:

Event Timeline

Slst2020 triaged this task as Medium priority.Mar 7 2023, 2:28 PM

check_for_entry() checks if a tool is already in our db, and does nothing if it is. If it's not, it inserts it into the tools table, unless the tool is deprecated

That's not quite right, as entry insertion happens independently of the deprecation check. So under the current workflow, a tool entry could be inserted into the db, and then that same tool could be found to be deprecated. Then what happens is that any existing, incomplete tasks associated with that tool are removed from the db (I leave completed tasks, as someone did bother to do the work and I wouldn't want them to be "punished" because a tool they happened to work on was deprecated), and that tool's journey is then over.

I want to maintain check_deprecation as it currently stands, primarily because of the "remove incomplete tasks" thing.

But I do agree about the uselessness of check_for_entry and have removed it and have replaced the add_new_entry function with an upsert function that all tools can be run through.

NicoleLBee changed the task status from Open to In Progress.Mar 7 2023, 3:40 PM
NicoleLBee claimed this task.
NicoleLBee moved this task from Backlog to In review on the Toolhunt board.

check_for_entry() checks if a tool is already in our db, and does nothing if it is. If it's not, it inserts it into the tools table, unless the tool is deprecated

That's not quite right, as entry insertion happens independently of the deprecation check. So under the current workflow, a tool entry could be inserted into the db, and then that same tool could be found to be deprecated. Then what happens is that any existing, incomplete tasks associated with that tool are removed from the db (I leave completed tasks, as someone did bother to do the work and I wouldn't want them to be "punished" because a tool they happened to work on was deprecated), and that tool's journey is then over.

Yes, that's desired behavior.

I want to maintain check_deprecation as it currently stands, primarily because of the "remove incomplete tasks" thing.

That's expected – this task does not mean to address check_deprecation, or anything that happens to tasks. The suggested is_deprecated function would be a first step toward untangling checking if a task is deprecated, from what happens to deprecated tasks.

That's expected – this task does not mean to address check_deprecation, or anything that happens to tasks. The suggested is_deprecated function would be a first step toward untangling checking if a task is deprecated, from what happens to deprecated tasks.

Ah, I see. I've added that change to the PR: d6e05e4