Page MenuHomePhabricator

Including scary transclusion contents in search results in CirrusSearch
Closed, InvalidPublic

Description

Feature summary: Currently, CirrusSearch does not index the contents of pages scarily transcluded onto the wiki. Maybe this should be possible?

Steps to reproduce:

  • Enable $wgEnableScaryTranscluding and CirrusSearch
  • Enable transclusion on the interwiki table for the mediawikiwiki prefix
  • Try to transclude {{mediawikiwiki::Help:Navigation}} on a page called Help:Navigation
  • Set $wgNamespacesToBeSearchedDefault[NS_HELP] = true;
  • Initialize the search index of CirrusSearch by running UpdateSearchIndexConfig.php
  • Index using ForceSearchIndex.php --skipLinks --indexOnSkip and then ForceSearchIndex.php --skipParse

Use case(s):

  • When people want to transclude content from other sources on the wiki, but also want to leverage CirrusSearch to search for the transcluded content on the wiki.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

@JeffreyWang: Hi, which use cases and problems would this solve? What are steps to reproduce?
(PS: Please use the templates linked in the preamble when creating tickets here - thanks.)

Sorry, I read too fast. I'll update the description with the template.

dcausse added a subscriber: dcausse.

Tentatively closing as Invalid as I could not reproduce the problem and was able to transclude interwiki templates and have its content indexed by Cirrus.
Perhaps the reasons it did not work for you are:

  • The config var to enable is wgEnableScaryTranscluding not wgScaryTransclusion
  • The interwiki you use must also be allowed for transclusion, please check the interwiki table entries and the iw_trans column

@dcausse Hi, thanks for looking into this! I should've been more clear with my reproduction steps, and I'll update these. I can confirm that both are enabled, otherwise I think it would be obvious that the page with the transclusion doesn't render properly. Did you try with a template or with a regular wiki page like my example?

@JeffreyWang I tested on a page on the main namespace named Test with the wikitext source {{mediawikiwiki::Help:Navigation}}:

Capture d’écran du 2021-07-14 15-15-25.png (477×1 px, 113 KB)

and then searched for some content and cirrus found it:

Capture d’écran du 2021-07-14 15-16-06.png (493×1 px, 72 KB)

Note that you can check what is inside the search index to verify if there are no discrepancies between what you see and what is indexed: simply add ?action=cirrusDump to the page URL to see a JSON dump of the elasticsearch document.

Huh, I have no idea why this didn't work before (job queue was empty) but I guess that resolves it. Very surprised, as I haven't changed anything since then, and I guess now it works. Maybe my search query was an edge case, no idea, can't remember it now. I'll try my best to be more descriptive in the future with the repro steps, appreciate your diligent help!