Page MenuHomePhabricator
Create Task
Maniphest T119644

Add option to exclude disambiguation pages from random/morelike (&others?) results.
Open, LowPublic
Actions

Description

For example, presently to show a single non-disambig random article we have to use one of the two following approaches:

  • add "pageprops" to the random query, run the query, check returned pageprops for "disambiguation" and if found keep re-requesting another random article until we get an article without "disambiguation" pageprop
  • make the random query fetch multiple random articles, say 5, and check for one without "disambiguation" pageprop, hoping that we didn't get 5 disambiguation articles

What would be nice to have is a parameter to exclude disambig pages, so, in the case of random, we could just ask for a single non-disambig random article.

Related Objects
Search...

Event Timeline

Mhurd created this task.Nov 25 2015, 6:34 PM
Mhurd raised the priority of this task from to Needs Triage.
Mhurd updated the task description. (Show Details)
Mhurd added a project: Discovery-ARCHIVED.
Mhurd added subscribers: Mhurd, • Niedzielski, • BGerstle-WMF, • Deskana.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 25 2015, 6:34 PM
• BGerstle-WMF added a comment.Nov 25 2015, 7:30 PM

Probably want to exclude redirects as well.

• BGerstle-WMF added a project: Wikipedia-iOS-App-Backlog.Nov 25 2015, 7:31 PM
• BGerstle-WMF set Security to None.
• BGerstle-WMF moved this task from Needs Triage to Tracking on the Wikipedia-iOS-App-Backlog board.
• BGerstle-WMF added a parent task: T113540: What can the Search API do for you?.
• BGerstle-WMF mentioned this in T113540: What can the Search API do for you?.Nov 25 2015, 7:34 PM
• BGerstle-WMF mentioned this in T118791: As a user exploring my feed I don't want to see disambiguation pages in random.Nov 25 2015, 10:04 PM
MaxSem edited projects, added MediaWiki-Action-API; removed Discovery-ARCHIVED.Nov 25 2015, 10:05 PM
Legoktm added a project: MediaWiki-extensions-Disambiguator.Nov 25 2015, 11:34 PM
• Deskana added a project: Discovery-ARCHIVED.Nov 25 2015, 11:46 PM

I've readded Discovery-ARCHIVED to this; since the task relates to content discovery mechanisms, this is definitely with Discovery's realm.

Anomie moved this task from Unsorted to Needs details or plan on the MediaWiki-Action-API board.Nov 26 2015, 1:36 AM
Anomie subscribed.Nov 26 2015, 1:47 AM

Note that the API querys for list=random are currently along the lines of

SELECT page_id, page_title, page_namespace, page_random, page_is_redirect FROM page WHERE page_random >= $START ORDER BY page_random, page_id LIMIT 5001
SELECT page_id, page_title, page_namespace, page_random, page_is_redirect FROM page WHERE page_namespace IN (...) AND page_random >= $START ORDER BY page_random, page_id LIMIT 5001
SELECT page_id, page_title, page_namespace, page_random FROM page WHERE page_is_redirect = $REDIR AND page_random >= $START ORDER BY page_random, page_id LIMIT 5001
SELECT page_id, page_title, page_namespace, page_random FROM page WHERE page_namespace IN (...)  AND page_is_redirect = $REDIR AND page_random >= $START ORDER BY page_random, page_id LIMIT 5001

SELECT page_id, page_title, page_namespace, page_random, page_is_redirect FROM page WHERE (page_random = $START AND page_id >= $STARTID OR page_random > $START) ORDER BY page_random, page_id LIMIT 5001
SELECT page_id, page_title, page_namespace, page_random, page_is_redirect FROM page WHERE page_namespace IN (...) AND (page_random = $START AND page_id >= $STARTID OR page_random > $START) ORDER BY page_random, page_id LIMIT 5001
SELECT page_id, page_title, page_namespace, page_random FROM page WHERE page_is_redirect = $REDIR AND (page_random = $START AND page_id >= $STARTID OR page_random > $START) ORDER BY page_random, page_id LIMIT 5001
SELECT page_id, page_title, page_namespace, page_random FROM page WHERE page_namespace IN (...)  AND page_is_redirect = $REDIR AND (page_random = $START AND page_id >= $STARTID OR page_random > $START) ORDER BY page_random, page_id LIMIT 5001

SELECT page_id, page_title, page_namespace, page_random, page_is_redirect FROM page WHERE (page_random = $START AND page_id >= $STARTID OR page_random > $START) AND page_random < $END ORDER BY page_random, page_id LIMIT 5001
SELECT page_id, page_title, page_namespace, page_random, page_is_redirect FROM page WHERE page_namespace IN (...) AND (page_random = $START AND page_id >= $STARTID OR page_random > $START) AND page_random < $END ORDER BY page_random, page_id LIMIT 5001
SELECT page_id, page_title, page_namespace, page_random FROM page WHERE page_is_redirect = $REDIR AND (page_random = $START AND page_id >= $STARTID OR page_random > $START) AND page_random < $END ORDER BY page_random, page_id LIMIT 5001
SELECT page_id, page_title, page_namespace, page_random FROM page WHERE page_namespace IN (...)  AND page_is_redirect = $REDIR AND (page_random = $START AND page_id >= $STARTID OR page_random > $START) AND page_random < $END ORDER BY page_random, page_id LIMIT 5001

You'll need to make sure that whatever changes you propose have decent performance when combined into these queries.

• Deskana triaged this task as Low priority.Dec 29 2015, 10:25 PM

This can't really be prioritised right now.

• Deskana moved this task from Needs triage to Search on the Discovery-ARCHIVED board.Dec 29 2015, 10:25 PM
Anomie merged a task: T126811: Remove disambiguation pages from the random API.Feb 12 2016, 9:08 PM
Anomie added a subscriber: kaldari.
Krinkle moved this task from Backlog to Improvements on the MediaWiki-extensions-Disambiguator board.Jul 20 2018, 2:20 AM
LGoto removed a project: Wikipedia-iOS-App-Backlog.Aug 27 2019, 8:08 PM
Restricted Application added a project: Platform Engineering. · View Herald TranscriptAug 27 2019, 8:08 PM
WDoranWMF removed a project: Platform Engineering.Aug 28 2019, 2:52 AM
Aklapper removed subscribers: Anomie, • BGerstle-WMF.Oct 16 2020, 5:02 PM
Bugreporter2 subscribed.Jan 13 2023, 11:09 AM

There are two ways to solve this:

  1. Using a bot, put everything into a category called category: index and then use Special: RandominCategory/index

Note Wikipedia doesn't have a proper index, which for an encyclopedia is ... odd.

  1. Make a disambiguation: namespace and remove disambiguation pages from the main namespace, where they have no business really.
Aklapper mentioned this in T347973: Option to Exclude Disambiguation Pages in Search API Endpoint.Oct 3 2023, 11:21 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL