Page MenuHomePhabricator
Create Task
Maniphest T111122

Improve completion suggester score function (by using page views to affect rankings)
Closed, ResolvedPublic
Actions

Description

1. Very high scores
Some pages have very high scores because :

  • they are massively linked in footers or headers:
    • Wikipedia
    • IP Address
    • Tilde
    • Copyright infringement
  • Digimon: unknown reason, will have to investigate.
  • List pages : large pages that link to each others
  • Some dates : incoming_link

This is mostly due to the initial scoring function, we expect that the addition of pageviews statistics in the score formula will help to mitigate these problems.

2. Scoring weirdness
white house will suggest white house farm murders first and not white house, again this is mostly due to our initial scoring algorithm. White house farm murders is a long page (longer than white house), is flagged as a quality page and has a lot of external links.
We hope that pageviews statistics will help to display more "obvious results".

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Integrate page views into completion scoremediawiki/extensions/CirrusSearchmaster+137 -46
Customize query in gerrit

Related Objects

Event Timeline

dcausse created this task.Sep 1 2015, 10:41 PM
dcausse raised the priority of this task from to Needs Triage.
dcausse updated the task description. (Show Details)
dcausse added projects: CirrusSearch, Discovery-ARCHIVED, OKR-Work.
dcausse subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 1 2015, 10:41 PM
dcausse mentioned this in T110915: Improve completion suggester precision.Sep 1 2015, 10:43 PM
dcausse renamed this task from Improve completion suggestion score function to Improve completion suggester score function.Sep 1 2015, 10:45 PM
dcausse set Security to None.
dcausse added a comment.Sep 1 2015, 10:50 PM

Pageviews data should be available with T44259

• Deskana subscribed.Dec 4 2015, 5:24 AM

This doesn't seem to be a blocker for T119535: EPIC: Create a Beta Feature for the new completion suggester so that users can give us feedback on it, but I'm guessing we will want to work on it after the beta feature is launched. @dcausse will know best.

Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptDec 4 2015, 5:24 AM
• Deskana moved this task from Needs triage to Search on the Discovery-ARCHIVED board.Dec 4 2015, 5:24 AM
• Deskana triaged this task as Medium priority.Dec 17 2015, 5:45 PM
• Deskana moved this task from Inbox to Scoring and result ordering on the CirrusSearch board.Dec 31 2015, 5:01 AM
• ksmith mentioned this in T121616: EPIC: Resolve technical or product blockers to enable us to do a more full rollout of the completion suggester.Jan 21 2016, 11:17 PM
dcausse merged a task: T120796: Integrate popularity score into completion suggester and full text search scoring.Jan 28 2016, 2:32 PM
dcausse added subscribers: gerritbot, EBernhardson.
dcausse claimed this task.Jan 28 2016, 2:34 PM
dcausse added a project: Discovery-Search (Current work).
dcausse moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.

Sorry, I wanted to close this one and merge it to T120796...
Adding this one to the sprint to reflect the work on T120796 .

dcausse moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Jan 28 2016, 3:05 PM
gerritbot added a comment.Jan 28 2016, 5:28 PM

Change 265771 had a related patch set uploaded (by EBernhardson):
Integrate page views into completion score

https://gerrit.wikimedia.org/r/265771

gerritbot added a project: Patch-For-Review.Jan 28 2016, 5:28 PM
EBernhardson added a comment.Jan 28 2016, 5:29 PM

One issue someone brought up yesterday, searching for 'obama' on http://en-suggesty.wmflabs.org/suggest.html provides, subjectively, much better results for prefix search than the completion suggester. Not sure what in particular to do about that.

• Deskana renamed this task from Improve completion suggester score function to Improve completion suggester score function (by using page views to affect rankings).Jan 28 2016, 5:42 PM
In T111122#1974004, @dcausse wrote:

Sorry, I wanted to close this one and merge it to T120796...
Adding this one to the sprint to reflect the work on T120796 .

It's all good! I tweaked the title of this one slightly to reflect the change and ongoing work.

dcausse added a comment.Jan 28 2016, 5:50 PM

Concerning Obama: "oboma" should be the top result thanks to the exact db match in prod. Concerning other suggestions I'll have a look.

EBernhardson moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Feb 2 2016, 5:20 PM
gerritbot added a comment.Feb 2 2016, 5:33 PM

Change 265771 merged by jenkins-bot:
Integrate page views into completion score

https://gerrit.wikimedia.org/r/265771

ReleaseTaggerBot added a project: MW-1.27-release (WMF-deploy-2016-02-02_(1.27.0-wmf.12)).Feb 2 2016, 6:00 PM
• Deskana closed this task as Resolved.Feb 3 2016, 6:20 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits