Page MenuHomePhabricator

Build Extra Plugin with extra-analysis-khmer and deploy to Maven Central
Closed, ResolvedPublic1 Estimated Story Points

Description

We need to re-build the plugin with the Khmer filter in it and deploy it to Maven Central so that it will be available to the Elasticsearch cluster machines.

After it is deployed, we'll need to install it on the cluster and then re-index. (see sub-tasks)

Event Timeline

Change 662923 had a related patch set uploaded (by DCausse; owner: DCausse):
[operations/software/elasticsearch/plugins@master] Add extra-analysis-khmer

https://gerrit.wikimedia.org/r/662923

Built the plugin using:
./mvnw -B -P deploy-central release:prepare && ./mvnw -B -P deploy-central,gpg release:perform
(activating the gpg profile should no longer be needed if we merge https://gerrit.wikimedia.org/r/c/wikimedia/discovery/discovery-parent-pom/+/662914)

Prepared a new debian package that needs to be built and deployed to our dpkg repo.

Moving to current work to finish this last step that requires admin privileges.

Gehel set the point value for this task to 1.Feb 15 2021, 4:12 PM

Change 662923 merged by Ryan Kemper:
[operations/software/elasticsearch/plugins@master] Add extra-analysis-khmer

https://gerrit.wikimedia.org/r/662923

The new debian package has been built and uploaded.

Steps followed to build/upload plugin after merging patch:

# Starting from plugins repo
# (1) Build locally and scp over to build host
./debian/rules prepare_build
cd ..
scp -r plugins/ deneb.codfw.wmnet:~

ssh deneb.codfw.wmnet
cd plugins
DIST=stretch-wikimedia pdebuild
exit

# (2) Transfer from build host to repo (apt) host, using local machine as intermediary
mkdir -p ~/wmf/elastic-plugins
scp 'deneb.codfw.wmnet:/var/cache/pbuilder/result/stretch-amd64/wmf-elasticsearch-search-plugins_6.5.4-5*' ~/wmf/elastic-plugins
scp -r ~/wmf/elastic-plugins apt1001.wikimedia.org:~/Elastic_Plugins/

ssh apt1001.wikimedia.org

# (3) Upload newly built package

GNUPGHOME=/root/.gnupg
REPREPRO_BASE_DIR=/srv/wikimedia
export GNUPGHOME
export REPREPRO_BASE_DIR

# replace `main` with `experimental` to deploy to experimental repo instead
# Check https://apt.wikimedia.org/wikimedia/pool/component/ to see if elastic65 is still the latest
# See https://wikitech.wikimedia.org/wiki/Reprepro#Building_an_unmodified_third-party_package_for_import for some general notes on the reprepro process
sudo -E reprepro -C component/elastic65 include stretch-wikimedia /home/ryankemper/Elastic_Plugins/wmf-elasticsearch-search-plugins_6.5.4-5_amd64.changes
rm -rfv ~/Elastic_Plugins

exit
rm -rfv ~/wmf/elastic-plugins
ssh deneb.codfw.wmnet rm -rfv ~/plugins

Now that the new debian package is built & uploaded, we can proceed to the actual roll-out (https://phabricator.wikimedia.org/T274204) when ready

Note: The initial build/upload was broken due to operator error, so we built/uploaded the new 6.5.4-6, which has been confirmed to work.

PLUGIN BUILD & UPLOAD STEPS PERFORMED:

# Starting from plugins repo
# (1) Build locally and scp over to build host
./debian/rules prepare_build
cd ..
ssh 'deneb.codfw.wmnet' 'sudo rm -rfv ~/plugins'

scp -r plugins/ deneb.codfw.wmnet:~

ssh deneb.codfw.wmnet
cd plugins
DIST=stretch-wikimedia pdebuild
exit

# (2) Transfer from build host to repo (apt) host, using local machine as intermediary
rm -rfv ~/wmf/elastic-plugins && mkdir -p ~/wmf/elastic-plugins
scp 'deneb.codfw.wmnet:/var/cache/pbuilder/result/stretch-amd64/wmf-elasticsearch-search-plugins_6.5.4-6*' ~/wmf/elastic-plugins
ssh 'apt1001.wikimedia.org' 'sudo rm -rfv ~/Elastic_Plugins' && scp -r ~/wmf/elastic-plugins apt1001.wikimedia.org:~/Elastic_Plugins/

# (3) Upload newly built package
ssh apt1001.wikimedia.org

GNUPGHOME=/root/.gnupg
REPREPRO_BASE_DIR=/srv/wikimedia
export GNUPGHOME
export REPREPRO_BASE_DIR


# replace `main` with `experimental` to deploy to experimental repo instead
# Check https://apt.wikimedia.org/wikimedia/pool/component/ to see if elastic65 is still the latest
# See https://wikitech.wikimedia.org/wiki/Reprepro#Building_an_unmodified_third-party_package_for_import for some general notes on the reprepro process
sudo -E reprepro -C component/elastic65 include stretch-wikimedia /home/ryankemper/Elastic_Plugins/wmf-elasticsearch-search-plugins_6.5.4-6_amd64.changes && sudo rm -rfv ~/Elastic_Plugins

exit
rm -rfv ~/wmf/elastic-plugins && ssh 'deneb.codfw.wmnet' 'sudo rm -rfv ~/plugins'