Page MenuHomePhabricator

Restore cross-wiki (interwiki) search across all public Wikimedia wikis
Open, MediumPublic

Description

We apparently already have the ability to do cross-wiki (or interwiki) searches using CirrusSearch/Elasticsearch, but the functionality has not yet been enabled everywhere. This task is to enable cross-wiki (or interwiki) search on all public Wikimedia wikis.

Event Timeline

MZMcBride raised the priority of this task from to Needs Triage.
MZMcBride updated the task description. (Show Details)
MZMcBride subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Nemo_bis renamed this task from Enable cross-wiki (interwiki) search across all public Wikimedia wikis to Restore cross-wiki (interwiki) search across all public Wikimedia wikis.Aug 22 2015, 9:30 PM
Nemo_bis triaged this task as Medium priority.
Nemo_bis edited projects, added Wikimedia-Search; removed MediaWiki-Search.
Nemo_bis set Security to None.
Nemo_bis subscribed.
Deskana lowered the priority of this task from Medium to Lowest.Dec 29 2015, 11:13 PM
Deskana subscribed.

Restoring inter-wiki search in its existing state (i.e. like you see it on the Italian Wikipedia) is low priority for Discovery-ARCHIVED. We are working on a number of inter-wiki, multi-language search projects, however.

Nemo_bis raised the priority of this task from Lowest to Medium.Dec 30 2015, 7:38 AM

This bug is a regression and as such fixing it is normal priority. Others can work on this.

It would be great if someone could clarify whether or not this is actually a regression. The title implies it is, as does one comment. But the description seems to say that nothing has changed, and instead that this is a request to extend existing functionality to more contexts.

i'm not familiar with interwiki search ever being enabled everywhere. The current implementation is a hack and our infrastructure cannot support enabling it everywhere. The tl/dr of the current implementation is it walks through a list of wiki's to query and issues a query for each individual wiki. Enabling the current interwiki search implementation would basically explode both response times and server load in the search cluster.

i'm not familiar with interwiki search ever being enabled everywhere.

It was in 2009, see T46420. Users have always liked the feature a lot.

I see that searching all wikis can be a huge performance issue.
As an alternative I would suggest to add interwiki links to the search result page that simply links to search result pages of other wikis without issuing a query on these wikis.

Hi,

Just to follow-up on this ticket, we launched sister project snippets (T145917) on the search results page (T162276) on June 15, 2017 and it is now available on all Wikipedias. However, enwiki has a different interface (fewer sister projects) as they had requested via an RfC (T162276#3278689) discussion.

Due to unforeseen issues, we don't have event logging publicly available yet, but we're working on getting the new dashboard statistics (T164854) into production soon.

Yesterday, we did a query of clickthrough traffic across all Wikipedias to get a simple snapshot idea of the usage of the sister projects (from the search results page) before and after the production release:

sister-projects_15June-26June2017.png (1×3 px, 409 KB)

Yesterday, we did a query of clickthrough traffic across all Wikipedias to get a simple snapshot idea of the usage of the sister projects (from the search results page) before and after the production release:

This is very useful! Wiktionary is an obvious first, but Wikinews is doing surprisingly well (compared to its traffic: good traffic for Wikisource, Wikiquote and Wikibooks is unsurpising). This suggests that cross-wiki search results are indeed covering queries which are inappropriate for Wikipedia (recent news, word definitions).

The absolute numbers are still low, but they'll need to be monitored in the longer term and I expect most impact to come from a self-sustaining virtuous circle as users learn about these additional resources.