Page MenuHomePhabricator
Create Task
Maniphest T174064

[FY 2017-18 Objective] Implement advanced search methodologies
Closed, InvalidPublic
Actions

Description

Implement advanced methodologies such as “learning to rank” machine learning techniques and signals to improve search result relevance across language Wikipedias.

Related Objects
Search...

StatusSubtypeAssignedTask
 InvalidNoneT174064 [FY 2017-18 Objective] Implement advanced search methodologies
 ResolvedEBernhardsonT161632 [Epic] Improve search by researching and deploying machine learning to re-rank search results
 ResolvedEBernhardsonT162053 backend data engineering and plumbing for LTRank
 ResolvedEBernhardsonT162054 Build oozie job to collect click data for learn to rank
 ResolvedEBernhardsonT162056 Generate DBN relevance labels from click data
 ResolvedEBernhardsonT162059 Kafka consumer to take learn to rank queries from a queue and run them against elasticsearch to generate relevance labels.
 ResolvedEBernhardsonT162072 Push LTRank feature generation queries into elasticsearch, and read back all the feature logs
 DuplicateEBernhardsonT162075 Oozie job for merging click data with DBN relevance scores
 ResolvedEBernhardsonT162311 Setup test/train/validation splits for our training data.
 ResolvedEBernhardsonT163407 Collect feature vectors from elasticsearch in mjolnir
 ResolvedEBernhardsonT166585 Calculate NDCG of click data that feeds the MLR pipeline
 ResolveddcausseT168813 Collect features from the sltr query
 ResolvedEBernhardsonT172589 Evaluate ensemble pruning algorithms
 ResolvedEBernhardsonT162061 Evaluate libraries for ranking with LambdaRank
 ResolveddcausseT162062 Specialize elasticsearch learning to rank plugin for our use case
 ResolvedEBernhardsonT162279 Collect ideas for feature engineering of LTRank
 ResolvedEBernhardsonT187148 Evaluate features provided by `query_explorer` functionality of ltr plugin
 ResolvedEBernhardsonT188015 Increase ltr.cache.max_size in Cirrus elasticsearch clusters
 ResolvedEBernhardsonT162369 Evaluate rescore windows for learning to rank
 ResolvedEBernhardsonT150032 Add support for interleaved results in 2-way A/B test
 ResolveddebtT171212 Interleaved results A/B test: turn on
 ResolvedEBernhardsonT171213 Interleaved results A/B test: check that data is flowing the way we expect
 ResolveddebtT171214 Interleaved results A/B test: turn off test
 ResolvedmpopovT171215 Interleaved results A/B test: analysis of data
 DeclinedEBernhardsonT171984 Turn on test of LTR with standard AB buckets and an interleaved bucket.
 ResolvedEBernhardsonT168664 Evaluate training set sizes for LTR models
 ResolvedEBernhardsonT169002 Load test the ltr-query plugin
 ResolvedTJonesT169103 Quick RelForge analysis of impact of initial LTR model
 ResolvedEBernhardsonT170009 Evaluate training speed and accuracy for 1M and 30M sample training sets with different worker counts
 ResolvedEBernhardsonT175771 Deploy AB test for all wikipedias with > 1% of search traffic
 ResolveddebtT177490 Stop A/B test on 18 wikis with > 1% of search traffic
 Resolved• chelsyxT176493 Analysis of testing on 18 wikis with > 1% of search traffic
 ResolvedEBernhardsonT175772 Deploy MLR as default content search to enwiki
 ResolveddcausseT175951 Search backend error during full_text search for 'QUERY_SRTING' after 39: i_o_exception: Can't read unknown type [50]
 DeclinedNoneT177520 Experiment with different grouping of queries that get fed into the DBN
 ResolveddcausseT177302 Relax the default AND of the retrieval query filter with minimum_should_match
 ResolveddebtT177502 Deploy A/B test to test relaxing the retrieval query filter
 ResolveddcausseT177956 Turn off: A/B test to test relaxing the retrieval query filter
 Resolved• chelsyxT177957 Analysis of: A/B test to test relaxing the retrieval query filter
 DeclinedNoneT177519 Build and AB test an ML Model with all the features exploded into individual pieces
 ResolvedEBernhardsonT182616 Re-run AB test for Hebrew Wikipedia (has > 1% of search traffic) with new model
 Resolved• chelsyxT183024 Analysis of hewiki's A/B test (> 1% of search traffic with a new model)
 OpenNoneT174705 Include phonetic search option to advanced / power user search
 OpenNoneT184771 Set up RelForge test of phonetic title search
 ResolvedTJonesT182708 Investigate effect of phonetic search on Wikipedia title words
 ResolvedGehelT171740 [Epic] Search Relevance: graded by humans
 ResolveddebtT171741 Search Relevance: MVP test (turned on)
 ResolveddebtT171742 Search Relevance: MVP test (turn it off)
 InvalidNoneT174106 Search Relevance Survey test #3: action items
 ResolvedEBernhardsonT174387 relevance survey: develop backend infrastructure to support lots of queries and lots of results per query
 ResolvedEBernhardsonT175046 Search Relevance Survey test #3: turn on test
 ResolvedEBernhardsonT175047 Search Relevance Survey test #3: turn off test
 ResolvedmpopovT175048 Search Relevance Survey test #3: analysis of test
 ResolvedmpopovT178096 Make a Puppet profile/role for doing R-based heavy stats/ML on Wikimedia Cloud
 DeclinedNoneT176278 Cleanup localstorage key after relevance survey's are complete
 InvalidNoneT176428 Search Relevance test #4 - action items
 DeclinedNoneT183027 Analysis: search relevance test #4
 InvalidNoneT178006 Search Relevance test #5: are users happy with the search results they got?
 DeclinedNoneT183028 Analysis: search relevance test #5

Event Timeline

• EBjune created this task.Aug 24 2017, 6:39 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 24 2017, 6:39 PM
• EBjune renamed this task from FY Objective 1: Implement advanced search methodologies to [FY 2017-18 Objective] Implement advanced search methodologies.Aug 24 2017, 6:43 PM
• EBjune triaged this task as Medium priority.Aug 24 2017, 6:56 PM
• EBjune added a project: Epic.
• EBjune added a subtask: T161632: [Epic] Improve search by researching and deploying machine learning to re-rank search results.Aug 31 2017, 4:38 PM
TJones created subtask T174705: Include phonetic search option to advanced / power user search.Aug 31 2017, 5:24 PM
• EBjune edited projects, added Goal, Discovery-Search; removed Epic.Sep 15 2017, 11:14 PM
TJones mentioned this in T176197: Allow hiragana searches to find katakana results and vice versa.Sep 19 2017, 8:49 PM
debt moved this task from needs triage to search-icebox on the Discovery-Search board.Sep 21 2017, 5:24 PM
Nemo_bis subscribed.Sep 28 2017, 4:46 PM
debt added a subtask: T171740: [Epic] Search Relevance: graded by humans.Oct 11 2017, 9:50 PM
TJones mentioned this in T191114: Implement personalized search for logged-in wiki users.Apr 6 2018, 4:05 PM
EBernhardson closed subtask T161632: [Epic] Improve search by researching and deploying machine learning to re-rank search results as Resolved.Sep 10 2018, 6:09 PM
Liuxinyu970226 subscribed.Sep 11 2018, 11:09 AM
Gehel closed subtask T171740: [Epic] Search Relevance: graded by humans as Resolved.Sep 9 2020, 2:51 PM
CBogen removed a project: Search-Platform-Programs.Oct 29 2020, 3:02 PM
CBogen closed this task as Invalid.Oct 29 2020, 3:26 PM
MPhamWMF closed subtask T174705: Include phonetic search option to advanced / power user search as Declined.Aug 1 2022, 9:21 PM
RhinosF1 reopened subtask T174705: Include phonetic search option to advanced / power user search as Open.Aug 2 2022, 1:32 PM
Restricted Application added a subscriber: RhinosF1. · View Herald TranscriptAug 2 2022, 1:32 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL