Page MenuHomePhabricator

Collect features from the sltr query
Closed, ResolvedPublic

Description

Currently feature collection in mjolnir runs a multi-search for each feature individually. Now that the ltr plugin is (or soon will be) available in prod and relforge it would be good to switch this over to collecting features from the ltr query as well.

Event Timeline

EBernhardson added a subscriber: dcausse.

@dcausse I know you were looking for somewhere to start in the mjolnir codebase, this might be as reasonable a place as any.

dcausse claimed this task.Jun 26 2017, 12:29 PM

I started to work on feature logging from the plugin, I think I'll continue on the plugin and use it in mjolnir.
Problem right now is that I can only obtain the explain output from a model, it means that a dummy model must be created before the fact. Logging inside the plugin should work with feature set and exisiting models.
So because both options (logging in the plugin vs explain) require some work in the plugin:

  • implement logging
  • extend sltr query to accept a featureset

I prefer to go with the long term approach and implement proper feature logging (unless there's some urgency in doing otherwise)?

debt added a subscriber: debt.Jun 26 2017, 4:28 PM

Proper feature logging sounds great to me, @dcausse :)

dcausse renamed this task from Collect features from the explain of an sltr query to Collect features from the sltr query.Jun 29 2017, 4:42 PM
dcausse updated the task description. (Show Details)

Feature logging in the plugin is written and can be reviewed: https://github.com/o19s/elasticsearch-learning-to-rank/pull/54
I'll now work on mjolnir.

Change 364433 had a related patch set uploaded (by DCausse; owner: DCausse):
[search/MjoLniR@master] Use a custom Accumulator to collect feature names

https://gerrit.wikimedia.org/r/364433

Change 364725 had a related patch set uploaded (by DCausse; owner: DCausse):
[search/MjoLniR@master] Extract feature vectors from the ltr plugin

https://gerrit.wikimedia.org/r/364725

Change 364433 merged by EBernhardson:
[search/MjoLniR@master] Use a custom Accumulator to collect feature names

https://gerrit.wikimedia.org/r/364433

Change 364725 merged by EBernhardson:
[search/MjoLniR@master] Extract feature vectors from the ltr plugin

https://gerrit.wikimedia.org/r/364725

debt closed this task as Resolved.Jul 28 2017, 7:17 PM