Page MenuHomePhabricator

Support querying "easy" translation recommendations
Closed, ResolvedPublic8 Estimated Story Points

Description

Along with all the existing recommendation API parameters, it should be possible to specify a difficulty level.

In this first phase, only access to easy recommendations is needed but the API interface should be designed in a way that is compatible with requesting other difficulty levels (medium, hard) in the future. For example: level: [1,2,3] or difficulty: [easy, medium, hard].

Recommendation difficulty level is initially based on size but could evolve to include aspects like unadapted templates, large tables, or other challenging wiki constructs.

For an article translation recommendation to be considered easy, the source article size should be between 2000 and 3000 kBs. For the purpose of this experiment, source articles smaller than 2000 kBs are considered stubs that are not worth translating.

For a section translation recommendation to be considered easy, the source section size should be between TBD and TBD kBs. (This should generally align with the quick translation label implemented in T360570 but the formula, 15 minutes at 200 words per minute, seams way too fast. Per that calculation, a 2000 kBs article would take 2 minutes to translate.)

Derived Requirements

  1. Ensure that the recommendation API supports a new parameter to filter results by difficulty level.
  2. Ensure that in the first phase, the API can return *easy* recommendations while keeping the design extensible to support other levels (medium, hard) in the future.
  3. Ensure that an article translation recommendation is classified as *easy* if the source article size is between 2000 and 3000 kBs.
  4. Ensure that source articles smaller than 2000 kBs are excluded from recommendations as they are considered stubs.
  5. Ensure that section translation recommendations for *easy* difficulty use a defined size range (currently TBD), aligned with the "quick translation" label logic.
  6. Ensure that the API query accepts both level: [1,2,3] and difficulty: [easy, medium, hard] formats for future compatibility.
Test Steps

Test Case 1: Ensure API Accepts Difficulty Parameter

  1. Send a request to the recommendation API with the parameter difficulty=easy.
  2. ✅❓❌⬜ AC1: Verify that the API returns only recommendations classified as *easy*.

QA Results - api.wikimedia

ACStatusDetails
1T399117#11108930

Event Timeline

ngkountas set the point value for this task to 8.Jul 17 2025, 1:39 PM
SBisson changed the task status from Open to In Progress.Jul 21 2025, 1:43 AM
SBisson claimed this task.
SBisson triaged this task as High priority.
SBisson moved this task from Prioritized to In-progress on the LPL Hypothesis board.

Change #1171281 had a related patch set uploaded (by Sbisson; author: Sbisson):

[mediawiki/services/cxserver@master] Include section sizes in section suggestion response

https://gerrit.wikimedia.org/r/1171281

Change #1171315 had a related patch set uploaded (by Sbisson; author: Sbisson):

[research/recommendation-api@master] Recommendations based on difficulty level

https://gerrit.wikimedia.org/r/1171315

@KCVelaga_WMF, @cchen do we have existing data to inform the size range an article or section has to be to be considered an "easy" translation?

at the moment I'm using article between 5kb and 25kb, and section between 1kb and 3kb but it is somewhat arbitrary.

Change #1171281 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Include section sizes in section suggestion response

https://gerrit.wikimedia.org/r/1171281

Change #1171315 merged by jenkins-bot:

[research/recommendation-api@master] Recommendations based on difficulty level

https://gerrit.wikimedia.org/r/1171315

Recommendation-API deployment expected this week.

Recommendation-API deployment expected this week.

There's a CX Server patch in this task that should be deployed as well to be able to check "easy" recommendations.

Change #1179243 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2025-08-14-134810-production

https://gerrit.wikimedia.org/r/1179243

Change #1179661 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update Recommendation API to 2025-07-25-064834-production

https://gerrit.wikimedia.org/r/1179661

Change #1179243 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2025-08-14-134810-production

https://gerrit.wikimedia.org/r/1179243

Change #1179661 merged by jenkins-bot:

[operations/deployment-charts@master] Update Recommendation API to 2025-07-25-064834-production

https://gerrit.wikimedia.org/r/1179661

Mentioned in SAL (#wikimedia-operations) [2025-08-19T13:35:07Z] <kart_> Updated Recommendation API to 2025-07-25-064834-production (T399117)

@SBisson Verified that the API returns only recommendations classified as *easy*, *medium*, *hard*, as seen from the links and screenshots below. I will move this to Sign-off. Thanks for all your work!

Test Result - api.wikimedia

Status: ✅ PASS
Environment: api.wikimedia
OS: macOS Sequoia 15.6.1
Browser: Chrome 138
Device: MBA
Emulated Device: NA

Test Artifact(s):

Test Steps

Test Case 1: Ensure API Accepts Difficulty Parameter

  1. Send a request to the recommendation API with the parameter difficulty=easy, difficulty=medium, difficulty=hard.
  2. AC1: Verify that the API returns only recommendations classified as *easy*, *medium*, *hard*.

easy
https://api.wikimedia.org/service/lw/recommendation/api/v1/translation?source=en&target=es&difficulty=easy&limit=3

2025-08-21_15-03-45.png (1,355×296 px, 139 KB)

medium
https://api.wikimedia.org/service/lw/recommendation/api/v1/translation?source=en&target=fr&difficulty=medium&limit=3

2025-08-21_15-04-07.png (1,391×315 px, 140 KB)

hard
https://api.wikimedia.org/service/lw/recommendation/api/v1/translation?source=en&target=fr&difficulty=hard&limit=3

2025-08-21_15-04-21.png (1,514×261 px, 111 KB)

GMikesell-WMF updated the task description. (Show Details)
GMikesell-WMF moved this task from Needs QA to Design Signoff on the LPL Hypothesis board.