Currently feature collection in mjolnir runs a multi-search for each feature individually. Now that the ltr plugin is (or soon will be) available in prod and relforge it would be good to switch this over to collecting features from the ltr query as well.
Description
Description
Details
Details
| Project | Branch | Lines +/- | Subject | |
|---|---|---|---|---|
| search/MjoLniR | master | +378 -20 | Extract feature vectors from the ltr plugin | |
| search/MjoLniR | master | +70 -21 | Use a custom Accumulator to collect feature names |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Invalid | None | T174064 [FY 2017-18 Objective] Implement advanced search methodologies | |||
| Resolved | EBernhardson | T161632 [Epic] Improve search by researching and deploying machine learning to re-rank search results | |||
| Resolved | EBernhardson | T162053 backend data engineering and plumbing for LTRank | |||
| Resolved | dcausse | T168813 Collect features from the sltr query |
Event Timeline
Comment Actions
@dcausse I know you were looking for somewhere to start in the mjolnir codebase, this might be as reasonable a place as any.
Comment Actions
I started to work on feature logging from the plugin, I think I'll continue on the plugin and use it in mjolnir.
Problem right now is that I can only obtain the explain output from a model, it means that a dummy model must be created before the fact. Logging inside the plugin should work with feature set and exisiting models.
So because both options (logging in the plugin vs explain) require some work in the plugin:
- implement logging
- extend sltr query to accept a featureset
I prefer to go with the long term approach and implement proper feature logging (unless there's some urgency in doing otherwise)?
Comment Actions
Feature logging in the plugin is written and can be reviewed: https://github.com/o19s/elasticsearch-learning-to-rank/pull/54
I'll now work on mjolnir.
Comment Actions
Change 364433 had a related patch set uploaded (by DCausse; owner: DCausse):
[search/MjoLniR@master] Use a custom Accumulator to collect feature names
Comment Actions
Change 364725 had a related patch set uploaded (by DCausse; owner: DCausse):
[search/MjoLniR@master] Extract feature vectors from the ltr plugin
Comment Actions
Change 364433 merged by EBernhardson:
[search/MjoLniR@master] Use a custom Accumulator to collect feature names
Comment Actions
Change 364725 merged by EBernhardson:
[search/MjoLniR@master] Extract feature vectors from the ltr plugin