Page MenuHomePhabricator

[epic] Test ChatGPT plugin output quality improvement
Closed, DeclinedPublic

Description

Based on qualitative coding of plugin output, we are seeing potential areas for improvement in increasing attribution quality, especially in non-English output (ChatGPT does not appear to follow our instructions for appending attribution language and links to every reply in non-EN).

  • @MPhamWMF & @Maryana to decide on goal for the sprint (e.g., increase attribution footer + link rate in $language(s) by X%)
  • Engineering implements tweaks to plugin prompt and uses Interrogator to test output
  • Eng + PM evaluate results & decide whether additional output quality tweaks may be promising to explore

Other ideas:
(Small = 1-2 days, medium = 3-4 , larger = 5-8)

  • Medium - Return more than first 12 paragraphs to chatgpt
  • Medium - Current events/news
  • Small - Internationalization / localization
    • Search the correct language wiki
  • Additional functionality beyond traditional full-wikipedia search
    • Small/Medium - Article-specific interactions/search
    • Small - Other API usage - eg contributor info etc
      • E.g # of watchers, # of contributors, last edited
    • Small - Disambiguation results
  • Medium - Chaining wiki calls