Page MenuHomePhabricator

API for history and diff of toolinfo records
Closed, ResolvedPublic

Description

Expose the versioned history of each toolinfo record via the API as well as tools to compare/revert/undo historical revisions.

  • GET /api/tools/{tool_name}/revisions/ - List all revisions of the given record
  • GET /api/tools/{tool_name}/revisions/{id}/ - Get revision details
  • GET /api/tools/{tool_name}/revisions/{id}/diff/{other_id}/ - Get a structured diff between two revisions
  • POST /api/tools/{tool_name}/revisions/{id}/revert/ - Restore the toolinfo record to the exact content of the given revision id.
  • POST /api/tools/{tool_name}/revisions/{id}/undo/{other_id}/ - Undo all changes made to the toolinfo record between {id} and {other_id}

Event Timeline

bd808 triaged this task as Medium priority.Jan 6 2021, 10:41 PM
bd808 updated the task description. (Show Details)
bd808 moved this task from Backlog to In Progress on the Toolhub board.

I have been thinking about how to represent a diff in an api response. The code in Striker produces an HTML rendering of a traditional text diff. This is also the approach taken by action=compare in the MediaWiki Action API. A rendered diff however does not seem like a great API response to me as it really doesn't provide structured data to the consumer.

Toolinfo data is structured rather than being free-form text. A Wikidata article is also a structured data collection, and they display diffs based on that structure rather than as free-form text diffs. This seems like the right way to go with toolinfo diffs too. I'm thinking the diff API response should be a list of add/remove/change actions against specific toolinfo fields. A change action could also carry a text diff of some kind, but as long as it carries both the "old" and "new" state that may be enough for a consuming tool. It does not appear that wikidata attempts to show insert or delete changes within a changed item itself.

I'm going to try implementing JSON Patch as part of the diff response, likely with the help of python-json-patch. This might even lead to implementing a PATCH endpoint for editing toolinfo if it all works out well.

Example output of /api/tools/{tool_name}/revisions/{id}/diff/{other_id}/ on a local test instance:

{
  "original": {
    "id": 358,
    "timestamp": "2021-01-27T20:45:36.708518Z",
    "user": null,
    "comment": "Initial version."
  },
  "operations": [
    {
      "op": "add",
      "path": "/available_ui_languages/1",
      "value": "fr"
    },
    {
      "op": "add",
      "path": "/available_ui_languages/2",
      "value": "pt-br"
    },
    {
      "op": "add",
      "path": "/available_ui_languages/3",
      "value": "he"
    },
    {
      "op": "add",
      "path": "/available_ui_languages/4",
      "value": "tr"
    },
    {
      "op": "add",
      "path": "/available_ui_languages/5",
      "value": "ko"
    },
    {
      "op": "add",
      "path": "/available_ui_languages/6",
      "value": "zh-hant"
    },
    {
      "op": "add",
      "path": "/available_ui_languages/7",
      "value": "mk"
    },
    {
      "op": "add",
      "path": "/available_ui_languages/8",
      "value": "diq"
    }
  ],
  "result": {
    "id": 384,
    "timestamp": "2021-01-27T23:07:46.935065Z",
    "user": {
      "id": 2,
      "username": "BDavis (WMF)"
    },
    "comment": "Imported from https://toolhub.toolforge.org/toolinfo.json"
  }
}

Change 663049 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[wikimedia/toolhub@main] api: edit history and diff endpoints for toolinfo records

https://gerrit.wikimedia.org/r/663049

Change 663049 merged by jenkins-bot:
[wikimedia/toolhub@main] api: edit history and diff endpoints for toolinfo records

https://gerrit.wikimedia.org/r/663049

After discussion with @srishakatux I think the remaining work here is to actually make some endpoints for revert/undo actions. And that discussion made it pretty clear that I also need to review MediaWiki's similar patrolling actions to make sure that we are using similar terminology and actions.

After discussion with @srishakatux I think the remaining work here is to actually make some endpoints for revert/undo actions. And that discussion made it pretty clear that I also need to review MediaWiki's similar patrolling actions to make sure that we are using similar terminology and actions.

I found https://www.mediawiki.org/wiki/Manual:Reverts which makes this research so much easier. Thank you @Ostrzyciel for your GSoC supported work on T258963: EditResult and reverted edits: write documentation on mediawiki.org!!

Based on the descriptions and terms used in Manual:Reverts, I think Toolhub should have two API actions:

  • POST /api/tools/{tool_name}/revisions/{id}/revert/ - Restore the toolinfo record to the exact content of the given revision id.
  • POST /api/tools/{tool_name}/revisions/{id}/undo/{other_id}/ - Undo all changes made to the toolinfo record between {id} and {other_id} by computing the diff between these two revisions (from id to other_id) and applying those changes to the current toolinfo record

We could also choose to provide a "rollback" action, but this seems at the moment to be a specialized action that can be functionally mimicked from the client side with the revert endpoint (restore to the most recent "good" version).

Change 664085 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[wikimedia/toolhub@main] api: revert and undo endpoints for toolinfo records

https://gerrit.wikimedia.org/r/664085

Change 664085 merged by jenkins-bot:
[wikimedia/toolhub@main] api: revert and undo endpoints for toolinfo records

https://gerrit.wikimedia.org/r/664085

bd808 moved this task from In Progress to Review on the Toolhub board.