Page MenuHomePhabricator

Exposing frequently searched terms for use by Wikipedia community
Open, LowPublic

Description

Please create a regularly updated report of frequent failed search terms to be used by Wikipedia community and/or publish the raw data for frequent search terms.

Purpose
Allow community optimization for search e.g:

  • Creating redirects for frequent search terms where article exist but with different name
  • Useful to understanding what are the most missing articles/a better alternative to Special:WantedPages which is sensitive to technical links (there is bias to links from templates/ navigation templates)
  • Doing it on regular basis may be useful for creating "wikitrends" like for searches

Notes

  • Security/Privacy concerns - Select some reasonable threshold on "most frequent" to avoid exposing private/sensitive data mistakenly searched.
    • If more than X unique readers search for some keyword it can be considered non private data.
    • Possible thresholds:
      • 10 unique readers search for a term in the same day
      • 50 unique readers search for a term in the same month
  • The purpose of this task is for community optimizations and not technical optimizations, so ideally no "technical searches" should be in the report - e.g "wiki/" prefix such as "Wiki/Barack Obama".
    • It is OK to somehow filter technical searches
    • It is more than OK to not filter them and let the users report bugs for technical optimizations for the search (continuing the example above: if someone search for "wiki/(.+)" it should be redirected to $1)
  • It is especially important for failed searches, since for successive search there is publicly published pagecounts views

Event Timeline

eranroz raised the priority of this task from to Needs Triage.
eranroz updated the task description. (Show Details)
eranroz subscribed.

So, here are some problems with gathering that data:

  1. There is no "X unique readers" data. We do not track uniques, and even a very entropic and brittle way of assessing uniques (using unique IP/UA tuples, say) requires data that isn't in the search logs. Very deliberately, our search logs contain no data that can be traced to individuals, and our logs that contain *that* data does not contain search information. Absent any way of sanitising, we would prefer not to surface this data because actually sanitising the search strings themselves is almost impossible.
  2. Honestly, having hand-reviewed the high-frequency requests, I don't see the community getting a tremendous amount of value out of this. When we discuss "failed" searches we mean searches that produced 0 results - and most of those that do, do for a reason. We're talking automata or queries that are nonsense text. There is work we can do on the search side (like detecting languages and translating them, or localising the results) but knowing that people search for 'Barack Obama' in Hindi on en.wikipedia does not necessarily mean a redirect can be justified (unfortunately!)
  3. The really high-frequency zero result requests are usually automata, not actual people.
Deskana changed the task status from Open to Stalled.Oct 15 2015, 4:29 PM
Deskana subscribed.

Marking as stalled on the basis of the comment by @Ironholds above. There's a lot of questions to resolve here.

  1. Even in this case there are other ways to sanitize the data. Other possible option: whitelist keywords that already appear as red links (or as links), e.g
select pl_title from pagelinks left join page on page_namespace =pl_namespace  and page_title =pl_title where page_title is null;
  1. It may or may not be justified to do manual redirect. If someone will go manually over such list he/she will probably find many other examples where redirect could be reasonable.

If keywords already exist as links, they most likely either (a) exist as pagetitles or (b) exist as terms within the article. In either case a result would be returned, and so it's not a failed query (in the sense of returning zero results)

It is not a technical fail of the search, but a fail for the reader - someone is looking for article that doesn't exist yet. Wikipedia community can use this data to prioritize articles for creation and anyway, it is interesting non sensitive data.

You're missing my point; suppose we include the words in the titles of redlinked pages as keywords, and we do not require that a search return 0 results to have failed. We then have several scenarios in the resulting data:

  1. A user searched for a redlinked page and didn't find anything. Success, from our point of view.
  2. A user searched for the topic covered by a redlinked page and actually did find what they were looking for in another article covered by the same keywords.
  3. A user searched for something largely unrelated to the topic, but the redlinked page has some of the same words in it, so their query gets in the list anyway and gives a false impression of which redlinks are most needed.
  4. A user searched for something largely unrelated to the topic, but the redlinked page has some of the same words in it, and the query also contained sensitive information we can't sanitise, so not only are we giving a false impression of redlink-filling necessity, we've also violated the privacy policy.

The only way of doing this to avoid that would be to require a query to *precisely match*, down to the phrasing and wording, the phrasing and wording of a redlink. Not only is this likely to be uncommon (Wikipedia's titling standards do not match standard human idioms for search, as our SEO results unfortunately reflect) it's also likely to be largely duplicative of looking at the pageviews for redlinked titles.

I don't know whether it is common or uncommon and whether the valid searches are representative sample or non representative sample of what the reader are interested in, but it would be nice to have even small chunk of the search data available to the public and this (red links white-listing) is very simple and clear method to sanitize. I'm not sure there is data of pageviews for redlinked titles, and if there is, I doubt it would match the same data for searches.

BTW it doesn't have to be precisely match raw data if the data isn't given as raw data but as processed/analyzed data:

  1. It is possible to score links based on number of their occurrences in searches -e.g "My password is Barack Obama" results in {"Barack Obama": 1, "Password": 1, "My":1, "is": 1 } etc . [Other way to think about it would be to build a search index not on titles but on pagelinks, and see what are the most common terms the search engine suggest]
  2. Co occurrence/ mutual information of substrings/words in terms appearing in other search terms would likely be a valid way to extract "native" terms from human idioms
debt claimed this task.
debt added subscribers: TJones, debt.

Hi!

I wanted to post a recent investigation into this general issue by @TJones - here's a snippet of it and the full analysis is here:

"I think the problem with all of these strategies is that so many high-frequency queries would be eliminated by any of them that any useful mining would be down to slogging through the low-impact long tail.

I don’t think there’s a lot here worth extracting, though others may disagree. The privacy concerns expressed earlier are genuine, and simple attempts to filter PII (using patterns, minimum IP counts, etc) are not guaranteed to be effective."

I'll close out this ticket as resolved.

The analysis is about zero results queries. This is different issue - a query may be in successful (page doesn't exist) but does have query results (Example: it is subsection of existing page).

If this is the case, and this subsection is popular, it may help WP community to split this subsection to an extended article