Page MenuHomePhabricator

Support robot indexing policies
Open, MediumPublic

Description

Default robot policies ($wgNamespaceRobotPolicies) (T122119) as well as __INDEX__ & __NOINDEX__ (T122252) are not applied in Flow boards.

Difference between Flow & other pages is that Flow content is in multiple pages: a topic is in the board, and in a Topic:xyz page.
I'm not aware of any possibility for partial search engine directives, where parts of a page are indexable and other are not, so we may have to support it only on a board level and let it trickle down to Topic:xyz pages?


Excerpt from Article::view, although we'll likely have to extract __INDEX__ & __NOINDEX__ from header descriptions ourselves still:

# Check for any __NOINDEX__ tags on the page using $pOutput
$policy = $this->getRobotPolicy( 'view', $pOutput );
$outputPage->setIndexPolicy( $policy['index'] );
$outputPage->setFollowPolicy( $policy['follow'] );

Event Timeline

matthiasmullie raised the priority of this task from to Needs Triage.
matthiasmullie updated the task description. (Show Details)
matthiasmullie subscribed.
Catrope renamed this task from Support noindex, nofollow to Support noindex.Aug 5 2015, 11:19 PM
Catrope triaged this task as Medium priority.Aug 5 2015, 11:19 PM
Mattflaschen-WMF renamed this task from Support noindex to Support robot indexing policies (tracking).Dec 22 2015, 10:53 PM
Mattflaschen-WMF updated the task description. (Show Details)

Can someone at least confirm to me that when a subject is marked as "hidden", it contains noindex html tag ? This case often appears in OTRS tickets.

Hah, hidden topics do not seem to have noindex (see this test case). We should fix that.

Hah, hidden topics do not seem to have noindex (see this test case). We should fix that.

I thought there would be an automatic comment or something, but apparently not: I've filed T184185: Hidden topics should be noindexed as a subtask of this task.

Change 403416 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/Flow@master] Implement robot-policy

https://gerrit.wikimedia.org/r/403416

SBisson renamed this task from Support robot indexing policies (tracking) to Support robot indexing policies.Jan 10 2018, 3:18 PM
SBisson claimed this task.