Page MenuHomePhabricator

Add Link engineering: Remove linkmodel.json from tmp-folder after retraining/updating link recommendation model
Closed, ResolvedPublic

Description

When generating link recommendations via querying the MySQL-tables, the model-file is written to the temp-folder (e.g. /tmp/simple.linkmodel.json), such that for subsequent link-recommendations calls for other articles in the same language, the model can be loaded directly without another MySQL-query. When we re-run the training-pipeline we add the updated model to the MySQL table, however, if the old model-file still exists in the tmp-file the DataetLoader will not query the new model but still use the old model.

Possible solution:

  • delete model file in tmp-folder when running the training-pipeline for a given language (not sure if recommended)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 656931 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[research/mwaddlink@main] training-pipeline: Remove link model from tmp if it exists

https://gerrit.wikimedia.org/r/656931

Change 656931 merged by jenkins-bot:
[research/mwaddlink@main] training-pipeline: Remove link model from tmp if it exists

https://gerrit.wikimedia.org/r/656931

Tgr subscribed.

Not really QA-able I suppose.