Page MenuHomePhabricator

Commons Caption Suggestion Tool — AI-assisted multilingual captions (human-in-the-loop)
Closed, DeclinedPublic

Description

Summary
Standalone tool to help Commons contributors add and improve multilingual image captions. It uses existing captions on a file, suggests translations in more languages via an LLM, and lets users edit and then save (or send) them to Commons. It is human-in-the-loop only: no automatic uploads or overwrites without review.

Current behavior

  • User pastes a Commons file URL; the tool loads the file’s current captions (structured data labels).
  • User selects target languages (defaults include en, es, fr, ar, zh); can add “favourite” languages (stored in browser).
  • For languages that already have a caption from Commons: user can edit and send updates.
  • For languages without a caption: user can “Generate” (single language) or “Generate all” to get translation suggestions from an existing caption using an LLM (now: OpenAI), then edit and send.
  • Captions are validated (short, factual, neutral) using custom validation rules before send. Saving uses the Commons API (wbsetlabel) with owner-only OAuth2 for now.

Stack

  • Frontend: Next.js, React, shadcn/ui.
  • Backend: Node.js, Express.
  • Translation/validation: OpenAI (text).
  • Deployment: Docker Compose (dev + production-style).

Possible follow-ups

  • Integrate into UploadWizard or other Commons flows.
  • Support more languages or batch workflows.
  • Replace owner-only OAuth with per-user OAuth for multi-user use.

Links

Event Timeline

@DSmit-WMF: Thanks for participating in the Hackathon! We hope you had a great time.

  • If this task was being worked on and resolved at the Hackathon: Please change the task status to resolved via the Add Action...Change Status dropdown, and make sure that this task has a link to the public codebase.
  • If this task is still valid and should stay open: Please add another active project tag to this task, so others can find this task (as likely nobody in the future will look back at the Hackathon workboard when trying to find something they are interested in).
  • In case there is nothing else to do for this task, or nobody plans to work on this task anymore: Please set the task status to declined.

Thank you,
Phabricator housekeeping service

No reply; setting task status to declined.