For optimal memory usage, the recommendation-api is using preprocessed wikidata_ids and decoded_lines_float32 numpy arrays as shown in T339890#9162780.
In T346218, we adapted the app to fetch and use these preprocessed numpy binaries from Swift.
The goal of this task is to upload both wikidata_ids.npy and decoded_lines_float32.npy to Swift.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T296994 Observations from research study for Section Translation on Thai Wikipedia | |||
| Open | None | T293648 Content Translation Recommendations API | |||
| Open | kevinbazira | T308164 Migrate Content Translation Recommendation API to Lift Wing | |||
| Open | kevinbazira | T339890 Host the recommendation-api container on LiftWing | |||
| Resolved | kevinbazira | T346218 Adapt the recommendation-api to use float32 preprocessed numpy arrays from swift | |||
| Resolved | kevinbazira | T346411 Upload recommendation-api preprocessed numpy binaries to Swift |
The recommendation-api preprocessed numpy binaries were uploaded successfully to Thanos Swift. Below are their storage URIs:
| file | uri |
| wikidata_ids.npy | s3://wmf-ml-models/recommendation_api/enwiki/20230915010659/wikidata_ids.npy |
| decoded_lines_float32.npy | s3://wmf-ml-models/recommendation_api/enwiki/20230915011054/decoded_lines_float32.npy |