Page MenuHomePhabricator

Include ElasticSearch (for Wikibase search) and related software as part of Wikibase release packages
Closed, ResolvedPublic13 Estimated Story Points

Description

As a Wikibase Release manager I want to be publish releases of the ElasticSearch and related software, so that Wikibase users can use the better search functionality easily on their installations.

Relevant Wikimedia custom Elasticsearch plugins:

  • org.wikimedia.search:extra
  • org.wikimedia.search.highlighter:experimental-highlighter-elasticsearch-plugin

Acceptance Criteria:

  • Version(s) of Elasticsearch that are compatible/required to be installed are indicated (despite this information potentially being available on https://www.mediawiki.org/wiki/Extension:CirrusSearch etc) in the results of the pipeline, so that users of tarball artifacts know what to install (Write a elasticsearch metadata file that can be used to produce release notes/documentation)
  • Uncertain: Compatible/required versions of the relevant Wikimedia plugins to Elasticsearch are included in a form of a tarball in the result of the "release pipeline"
  • Compatible and recommended version of Elasticsearch (including relevant Wikimedia plugins) is included in a form of the Docker container image in the result of the "release pipeline"
  • Relevant Mediawiki extensions are included in the tarball result of the "release pipeline" in versions which is compatible with the included version of Wikibase
    • Elastica
    • CirrusSearch
    • WikibaseCirrusSearch
  • Relevant Mediawiki extensions are included in the Docker container image of the "release pipeline" in versions which is compatible with the included version of Wikibase
    • Elastica
    • CirrusSearch
    • WikibaseCirrusSearch

Notes

  • "in a version which is compatible with the included version of Wikibase" in the acceptance criteria above means that automated testing of integration between the relevant extensions and the released version of Wikibase has happened during the releasing process
  • Official elasticsearch Docker images (https://www.docker.elastic.co/r/elasticsearch) might be of use here

Event Timeline

Compatible and recommended version of Elasticsearch is included in a form of the Docker container image in the result of the "release pipeline"

Should we note that the relevant plugins should also be included? or have they been excluded for some reason?
The dependencies are there although perhaps not clearly documented?

Official elasticsearch Docker images (https://hub.docker.com/_/elasticsearch) might be of use here

Just a note but this is a mirror, as the main location for the official images is https://www.docker.elastic.co/r/elasticsearch seemingly.

It seems to me we might need to determine exactly the needs of the plugin stuff.

We have some options:

  • Release the plugin and specify which elasticsearch versions work with both wikibase and the plugin
  • release a working elasticsearch and plugin combined
  • Stop relying on the plugin? I'm totally ill informed but is this still really needed? Highlighting has been in ES for a while (TM) and I'm not sure if we really need all the extra features this plugin offers for wikibase

Yes, I'll adjust the description with the specific details. Not sure if this should stop estimation etc

Plugins are required for language support but also for specific features required by wikibase (e.g. the custom highlighter we use allows to distinguish between the label and aliases).
Note that we build a debian package for the required elasticsearch plugins (wmf-elasticsearch-search-plugins_6.5.4-4~stretch_all.deb from https://apt.wikimedia.org/wikimedia/dists/stretch-wikimedia/component/elastic65/binary-amd64/)

WMDE-leszek renamed this task from Add ElasticSearch (for Wikibase search) and related software to Wikibase "release pipeline" to Release ElasticSearch (for Wikibase search) and related software as part of Wikibase release packages.Jan 19 2021, 8:22 PM
WMDE-leszek renamed this task from Release ElasticSearch (for Wikibase search) and related software as part of Wikibase release packages to Include ElasticSearch (for Wikibase search) and related software as part of Wikibase release packages.Jan 19 2021, 8:57 PM
Tarrow set the point value for this task to 13.Jan 21 2021, 10:28 AM
toan removed the point value 13 for this task.
toan set the point value for this task to 13.
Tarrow moved this task from Doing to To do on the Wikibase Release Strategy (Sprint 5) board.
Tarrow moved this task from To do to Doing on the Wikibase Release Strategy (Sprint 5) board.