**Status** approved by Tim Startling in [[ https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-03-04-21.00.html | 2015-03-04 IRC RFC discussion]], requesting some followup (see task comments).
It would be useful to give the ContentHandler (resp. the Content object) for a specific content model the ability to control what fields are exposed to the search engine index. For wikitext, this would be the main "text" field, but could also include "html" for rendered html, "links" for outgoing links, etc. For Wikibase entities, this would include fields like "label" (which would be a multi-lingual field), "alias", "description", "sitelink", "property-value", etc.
Currently, Content::getTextForSearchIndex() exposes flat text to the search index, assuming word based full text indexing is applied.
This RFC proposes to add the following:
`Content::getFieldsForSearchIndex()`: This would return an associative array mapping field names to index values. The type and structure of the index value must correspond to the type of the field as declared by getSearchIndexFieldDefinitions, see below. At least the "text" field should be returned. It would be populated by calling the old getTextForSearchIndex() method.
`ContentHandler::getSearchIndexFieldDefinitions()`: This returns a list of SearchIndexFieldDefinition objects, representing the fields that Content::getFieldsForSearchIndex() may return for the handler's content model. This information should be used by the search engine when defining indexes. TextContent would implement this to return a definition for the "text" field, defining it to be plain text eligible for word-based full text indexing.
`ContentHandler::getAllSearchIndexFieldDefinitions()`: Static methiod that calls getSearchIndexFieldDefinitions() on all registered content handlers, and combines the results. If two content handlers declare the same fields with a different type, an exception is thrown.
`class SearchIndexFieldDefinition`: This is a value object with the following methods:
* `getName()`: returns the field name. Fields with the same name may be used by different content models, but they must have the same declaration.
* `getIndexType()`: returns the index type (see below)
* `isMultiValue()`: returns true if the field is a list of values of the said type.
* `getWeight()`: (between 0 and 99? or between 0 and 1? Or between 0 and PHP_INT_MAX?) (wouldn't recommend PHP_INT_MAX, since it can vary by build and may change in the future).
Index types:
* INDEX_TYPE_TEXT: String. Allow (word based) full text search if possible.
* INDEX_TYPE_MULTILINGUAL: Associative array of language code mapping to INDEX_TYPE_TEXT values. Allow (word based) full text search if possible.
* INDEX_TYPE_IDENTIFIER: String. Allow prefix matches if possible.
* INDEX_TYPE_QUANTITY: Signed float. Allow range queries of possible.
* INDEX_TYPE_GEOPOINT: A pair of longitude and latitude, represented as floats. Allow special queries if feasible.
* INDEX_TYPE_DATETIME: A timestamp (in a format wfTimestamp understands). Allow range queries if possible.
Search engines may ignore fields that have unsupported types, or may treat values of such types as plain strings or text.
So, for text, TextContentHandler::getSearchIndexFieldDefinitions() would return
array(
new SearchIndexFieldDefinition( 'text', INDEX_TYPE_TEXT )
)
And TextContent::getFieldsForSearchIndex() would return
array(
'text' => $this->getTextForSearchIndex()
)
(Alternatively, `getTextForSearchIndex()` would call `getFieldsForSearchIndex()`)
TBD: In the next step, SearchEngine should be modified to make use of the new information. In particular, `SearchEngine::getTextFromContent` should be deprecated, and replaced by a `getFieldsFromContent` method.
TBD: To make full use of having multiple fields indexed for search, these fields should be accessible in the SearchResult. This ties in with Brion's proposal for `SearchResult::getMetadata()` T78011.
TBD: We may want to expose a "widget type" or "data type" that can be used to pick formatters or widgets for showing or inputting values for a field. These types would be related to, but distinct from, the index types.