Page MenuHomePhabricator
Create Task
Maniphest T283860

Another round of search tuning based on the results from image-recommendation-test
Closed, ResolvedPublic
Actions

Description

We have an extra ~2.5k rated search results from image-recommendation-test, so we should probably re-tune our search results with these added to our training dataset.

erm ... I (@Cparle) probably will have to do this, seeing as I did the original search tuning and can re-do it without too much effort, but in the process I ought to make the process easier for anyone to do

Acceptance criteria:
[] a new set of parameters for search signals to be used in the search profile

  • an easier to follow process to generate those parameters

Related Objects
Search...

Event Timeline

Cparle created this task.May 27 2021, 7:12 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 27 2021, 7:13 PM
Cparle added a project: Structured-Data-Backlog.May 27 2021, 7:13 PM
CBogen edited projects, added Structured-Data-Backlog (Current Work); removed Structured-Data-Backlog.Jun 14 2021, 4:55 PM
CBogen moved this task from Incoming to Ready for Estimation on the Structured-Data-Backlog (Current Work) board.Jun 14 2021, 5:04 PM
CBogen moved this task from Ready for Estimation to Doing on the Structured-Data-Backlog (Current Work) board.Jun 30 2021, 4:22 PM
Cparle added a parent task: T280368: [XL] Get a sense of how much labeled data we need by plotting learning curves.Jun 30 2021, 4:23 PM
Cparle added a comment.Jul 12 2021, 4:28 PM

This will get done as part of T280368

Cparle claimed this task.Jul 12 2021, 4:29 PM
Cparle updated the task description. (Show Details)Jul 22 2021, 11:47 AM
Cparle closed this task as Resolved.Jul 22 2021, 11:51 AM

So the easier-to-follow process for doing search tuning is

  1. https://github.com/cormacparle/media-search-signal-test#how-to-create-a-dataset-in-ranklib-format-quick-version
  2. https://github.com/cormacparle/media-search-signal-test#3-train-a-logistic-regression-model-using-ranklib

Tuning the results using the new data actually gives us a slightly worse precision@25 over all the labeled data than we have with the current tuned search (0.82 instead of 0.83) so I propose we keep what we have already

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL