We need to re-build the plugin with the Khmer filter in it and deploy it to Maven Central so that it will be available to the Elasticsearch cluster machines.
After it is deployed, we'll need to install it on the cluster and then re-index. (see sub-tasks)
| Project | Branch | Lines +/- | Subject | |
|---|---|---|---|---|
| operations/software/elasticsearch/plugins | master | +57 -45 | Add extra-analysis-khmer |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Invalid | None | T218613 {EPIC} Search: Local Impact. Making making bigger improvements for smaller (or underrepresented) communities | |||
| Resolved | TJones | T185721 Null or inconsistent search results using Khmer script | |||
| Resolved | RKemper | T274203 Build Extra Plugin with extra-analysis-khmer and deploy to Maven Central | |||
| Open | RKemper | T274204 Deploy new version of Extra Plugin (with Khmer filter) to Elasticsearch cluster | |||
| Resolved | TJones | T274205 Reindex Khmer wikis to enable Khmer syllable reordering | |||
| Resolved | EBernhardson | T275975 Search broken on beta cluster wikis |
Change 662923 had a related patch set uploaded (by DCausse; owner: DCausse):
[operations/software/elasticsearch/plugins@master] Add extra-analysis-khmer
Built the plugin using:
./mvnw -B -P deploy-central release:prepare && ./mvnw -B -P deploy-central,gpg release:perform
(activating the gpg profile should no longer be needed if we merge https://gerrit.wikimedia.org/r/c/wikimedia/discovery/discovery-parent-pom/+/662914)
Prepared a new debian package that needs to be built and deployed to our dpkg repo.
Moving to current work to finish this last step that requires admin privileges.
Change 662923 merged by Ryan Kemper:
[operations/software/elasticsearch/plugins@master] Add extra-analysis-khmer
The new debian package has been built and uploaded.
Steps followed to build/upload plugin after merging patch:
# Starting from plugins repo # (1) Build locally and scp over to build host ./debian/rules prepare_build cd .. scp -r plugins/ deneb.codfw.wmnet:~ ssh deneb.codfw.wmnet cd plugins DIST=stretch-wikimedia pdebuild exit # (2) Transfer from build host to repo (apt) host, using local machine as intermediary mkdir -p ~/wmf/elastic-plugins scp 'deneb.codfw.wmnet:/var/cache/pbuilder/result/stretch-amd64/wmf-elasticsearch-search-plugins_6.5.4-5*' ~/wmf/elastic-plugins scp -r ~/wmf/elastic-plugins apt1001.wikimedia.org:~/Elastic_Plugins/ ssh apt1001.wikimedia.org # (3) Upload newly built package GNUPGHOME=/root/.gnupg REPREPRO_BASE_DIR=/srv/wikimedia export GNUPGHOME export REPREPRO_BASE_DIR # replace `main` with `experimental` to deploy to experimental repo instead # Check https://apt.wikimedia.org/wikimedia/pool/component/ to see if elastic65 is still the latest # See https://wikitech.wikimedia.org/wiki/Reprepro#Building_an_unmodified_third-party_package_for_import for some general notes on the reprepro process sudo -E reprepro -C component/elastic65 include stretch-wikimedia /home/ryankemper/Elastic_Plugins/wmf-elasticsearch-search-plugins_6.5.4-5_amd64.changes rm -rfv ~/Elastic_Plugins exit rm -rfv ~/wmf/elastic-plugins ssh deneb.codfw.wmnet rm -rfv ~/plugins
Now that the new debian package is built & uploaded, we can proceed to the actual roll-out (https://phabricator.wikimedia.org/T274204) when ready
Note: The initial build/upload was broken due to operator error, so we built/uploaded the new 6.5.4-6, which has been confirmed to work.
PLUGIN BUILD & UPLOAD STEPS PERFORMED:
# Starting from plugins repo # (1) Build locally and scp over to build host ./debian/rules prepare_build cd .. ssh 'deneb.codfw.wmnet' 'sudo rm -rfv ~/plugins' scp -r plugins/ deneb.codfw.wmnet:~ ssh deneb.codfw.wmnet cd plugins DIST=stretch-wikimedia pdebuild exit # (2) Transfer from build host to repo (apt) host, using local machine as intermediary rm -rfv ~/wmf/elastic-plugins && mkdir -p ~/wmf/elastic-plugins scp 'deneb.codfw.wmnet:/var/cache/pbuilder/result/stretch-amd64/wmf-elasticsearch-search-plugins_6.5.4-6*' ~/wmf/elastic-plugins ssh 'apt1001.wikimedia.org' 'sudo rm -rfv ~/Elastic_Plugins' && scp -r ~/wmf/elastic-plugins apt1001.wikimedia.org:~/Elastic_Plugins/ # (3) Upload newly built package ssh apt1001.wikimedia.org GNUPGHOME=/root/.gnupg REPREPRO_BASE_DIR=/srv/wikimedia export GNUPGHOME export REPREPRO_BASE_DIR # replace `main` with `experimental` to deploy to experimental repo instead # Check https://apt.wikimedia.org/wikimedia/pool/component/ to see if elastic65 is still the latest # See https://wikitech.wikimedia.org/wiki/Reprepro#Building_an_unmodified_third-party_package_for_import for some general notes on the reprepro process sudo -E reprepro -C component/elastic65 include stretch-wikimedia /home/ryankemper/Elastic_Plugins/wmf-elasticsearch-search-plugins_6.5.4-6_amd64.changes && sudo rm -rfv ~/Elastic_Plugins exit rm -rfv ~/wmf/elastic-plugins && ssh 'deneb.codfw.wmnet' 'sudo rm -rfv ~/plugins'