In T191114#4109926, @Multichill wrote:

In T191114#4109676, @debt wrote:

Hi @Multichill - this is a very interesting idea, but it goes against all of our privacy concerns with the Wikipedia community and the movement in general. I believe that this idea has come up several times over the years, but because of privacy concerns, nothing has been done with it (marking as declined).

That's just FUD. Can you provide links where this has been discussed before? Can you explain exactly what the privacy concerns are? I'm not buying a declined without a decent conversation about this.

Hi @Multichill - here's several items to read through. They don't all necessarily exactly match what you're proposing, but they're all pretty close.

• Data retention policy (always a good place to start): https://meta.wikimedia.org/wiki/Data_retention_guidelines
• Gather (a failed experiment on collecting, recommending, and publishing links to articles): https://www.mediawiki.org/wiki/Gather
• Finding searches with zero results (link is to a comment from @Deskana): https://phabricator.wikimedia.org/T8373#1856036
• Conclusion on the usefulness of publishing searches with zero results and overall search related privacy concerns (written by @TJones): https://www.mediawiki.org/wiki/User:TJones_(WMF)/Notes/Top_Unsuccessful_Search_Queries#Conclusions

It's true that publishing zero-result searches (or any searches) is a more obvious potential privacy leak than personalization, because the data for personalization should be hidden away internally. However, if there ever were a data breach of any kind, personalization data would be a concentrated source of information on users. I'm not an ops engineer, but storing and securing personalized data properly for hundreds of thousands to millions of users could entail more hardware, and seems likely to require more general complexity in our infrastructure.

Good personalization would require a fair amount of data for each user, and might have to go beyond our 90-day data retention policy to be effective. Users could opt in, but it's not clear that we want to even give people the option to violate the data retention policy. Given the importance of privacy—otherwise why not use site search from Google?—it seems like this kind of data retention (even within the 90 day limit) would require opt-in, which could significantly limit its use.

As you mentioned, this is a very high level task that would require a lot of work, which means it has a high opportunity cost and would require prioritization over other ongoing projects, which currently include supporting search for Structured Data on Commons, improving our new machine learning–based ranking (T174064), improving search in multiple languages (T174065), paying off a lot of technical debt for our query parser (T185108) as prelude for general improvements there, and supporting Wikidata and Wikibase search (T189736, etc.). You can see our goals (and goals for the whole Technology Department) on MediaWiki—our team's goals are here and our support for Structured Data on Commons is here.

So, the conversation I'd like to see would be something like what happens during the Community Wishlist Survey—though the next one is a ways away—or a proposal on a relevant Village Pump (I checked English Wikipedia's VP and couldn't find anything relevant). It would require a lot of conversation to come up with a set of desirable use cases that give the right balance of privacy and features. I fear additional complications when trying to plan for something across communities, too. Sometimes features that seem generically and obviously beneficial to one groups can become contentious within a different community, and something like this would require broad consensus across many wiki communities. Users being able to individually disable it may not be enough for some communities; I can't give their arguments because I don't know them, but I've come to expect them—and for something as ambitious and broad as this, I assume there could be significant pushback in some quarters.

As it stands, I have concerns about privacy, infrastructure, usage uptake, community acceptance, consensus on scope, and opportunity costs—and that's before getting into the actual technical implementation details, which are also potentially worrisome. We are constrained in our ability to gather search quality data—Google and Bing pay big bucks for it—we already have difficulty assessing some kinds of changes, especially on smaller wikis. Assessing personalized search results could be very difficult, and building something so complex without proper feedback is risky.

Personalized search would be a huge undertaking and it needs much more than one phab ticket to set it underway.

Declining this after re-reading the above comments regarding privacy, etc