Page MenuHomePhabricator
Create Task
Maniphest T306422

Upgrade translatewiki elasticsearch version to 6.8.23
Closed, ResolvedPublic4 Estimated Story Points
Actions

Description

The elasticsearch cluster running behind translatewiki.net should be upgraded to 6.8.23. Reasons are:

Note that after upgrading to 6.8 all the indices should be re-indexed so that they pick-up the most recent default index settings/mappings.

Details

SubjectRepoBranchLines +/-
ttmserver-export: Add workaround for T258860mediawiki/extensions/Translatemaster+5 -0
Fixes for Elasticsearch upgradetranslatewikimaster+2 -1
Update Elasticsearch to 6.8.23translatewikimaster+24 -37
Customize query in gerrit

Related Objects
Search...

Event Timeline

dcausse created this task.Apr 19 2022, 8:52 AM
Restricted Application added a project: Discovery-Search. · View Herald TranscriptApr 19 2022, 8:52 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Nikerabbit subscribed.Apr 19 2022, 2:12 PM

@dcausse (or anyone): do you have a timeline for WMF upgrades? I am considering this task for May, but if it blocks your work (or there is other good reason) then I would prioritize this earlier.

dcausse added a comment.Apr 19 2022, 2:33 PM

@Nikerabbit ideally we would like to have migrated all MW extensions depending on elasticsearch by may 13th so addressing this task in the first half of May would be ideal for us.

Nikerabbit triaged this task as High priority.Apr 19 2022, 2:48 PM
Nikerabbit added projects: Language-Team (Language-2022-April-June), Unplanned-Sprint-Work.
Nikerabbit added a subscriber: abi_.

Thanks. Because we have an off-site coming up, I'm putting this in the current sprint already. @abi_ can you estimate this task?

Nikerabbit moved this task from Quarter Backlog to Priority: Localization infrastructure on the Language-Team (Language-2022-April-June) board.Apr 19 2022, 2:48 PM
Gehel moved this task from needs triage to Current work on the Discovery-Search board.Apr 20 2022, 7:50 PM
Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.
MPhamWMF moved this task from Incoming to Waiting on the Discovery-Search (Current work) board.Apr 20 2022, 7:50 PM
abi_ set the point value for this task to 4.Apr 25 2022, 12:33 PM
abi_ claimed this task.Apr 26 2022, 4:16 AM
abi_ moved this task from Priority: Localization infrastructure to In Progress on the Language-Team (Language-2022-April-June) board.

I'm running Elasticsearch 6.5.4 on my machine and I'll be updating it to 6.8.23

gerritbot added a comment.Apr 28 2022, 11:05 AM

Change 787467 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] Update Elasticsearch to 6.8.23

https://gerrit.wikimedia.org/r/787467

gerritbot added a project: Patch-For-Review.Apr 28 2022, 11:05 AM
abi_ added a comment.Apr 28 2022, 11:17 AM

I updated my local instance Elasticsearch from 6.5.4 to 6.8.23

  1. Disable translation memory by removing it from settings
  2. Remove Elasticsearch 6.5.4 by running sudo apt-get purge elasticsearch
  3. Grab Elasticsearch 6.8.23 deb file from https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.8.23.deb
  4. Install: sudo dpkg -i elasticsearch-6.8.23.deb
  5. Enable auto-start: sudo /bin/systemctl enable elasticsearch.service
  6. Start Elasticsearch: sudo systemctl start elasticsearch.service
  7. Install wikimedia-search-extra plugin: In /usr/share/elasticsearch run sudo ./bin/elasticsearch-plugin install org.wikimedia.search:extra:6.8.23-wmf1
  8. Install search-highlighter plugin: In /usr/share/elasticsearch run sudo ./bin/elasticsearch-plugin install org.wikimedia.search.highlighter:experimental-highlighter-elasticsearch-plugin:6.8.23
  9. Restart Elasticsearch - sudo systemctl restart elasticsearch.service
  10. Re-enable translation memory again
  11. Run php extensions/Translate/scripts/ttmserver-export.php from the MediaWiki core directory to bootstrap the translation memory

I did not see any issues, the update went smoothly, and I am getting suggestions from translation memory.

Gehel moved this task from Waiting to Blocked/Waiting on the Discovery-Search (Current work) board.May 9 2022, 3:16 PM
Nikerabbit added a subtask: T307998: Unable to run ttmserver-export.php: Container disabled!.May 10 2022, 10:50 AM
Nikerabbit added a subscriber: Gehel.May 10 2022, 10:53 AM

@dcausse @Gehel Heads up that we may be delayed on completing this due to T307998: Unable to run ttmserver-export.php: Container disabled!.

abi_ reassigned this task from abi_ to Nikerabbit.May 10 2022, 10:55 AM
Nikerabbit moved this task from In Progress to Blocked on the Language-Team (Language-2022-April-June) board.May 10 2022, 10:59 AM
Nikerabbit moved this task from Blocked to In Progress on the Language-Team (Language-2022-April-June) board.May 17 2022, 8:59 AM
gerritbot added a comment.May 18 2022, 1:37 PM

Change 787467 merged by jenkins-bot:

[translatewiki@master] Update Elasticsearch to 6.8.23

https://gerrit.wikimedia.org/r/787467

dcausse mentioned this in T308676: Elasticsearch 7.10.2 rollout plan.May 18 2022, 2:17 PM
Maintenance_bot removed a project: Patch-For-Review.May 18 2022, 2:30 PM
Nikerabbit moved this task from In Progress to Done on the Language-Team (Language-2022-April-June) board.May 18 2022, 8:14 PM

We had a few difficulties but we managed to complete the upgrade and reindex the search index and the translation search/memory index.

https://translatewiki.net/wiki/Special:Version

gerritbot added a comment.May 19 2022, 4:00 AM

Change 793062 had a related patch set uploaded (by Abijeet Patro; author: Nikerabbit):

[translatewiki@master] Fixes for Elasticsearch upgrade

https://gerrit.wikimedia.org/r/793062

gerritbot added a project: Patch-For-Review.May 19 2022, 4:00 AM
gerritbot added a comment.May 19 2022, 7:16 AM

Change 793393 had a related patch set uploaded (by Nikerabbit; author: Nikerabbit):

[mediawiki/extensions/Translate@master] ttmserver-export: Add workaround for T258860

https://gerrit.wikimedia.org/r/793393

Nikerabbit moved this task from Backlog to System admin stuff on the translatewiki.net board.May 19 2022, 7:49 AM
bking subscribed.May 26 2022, 6:38 PM
gerritbot added a comment.May 30 2022, 10:44 AM

Change 793062 merged by jenkins-bot:

[translatewiki@master] Fixes for Elasticsearch upgrade

https://gerrit.wikimedia.org/r/793062

dcausse added a comment.May 30 2022, 3:13 PM

@abi_ can we consider this task done and close it so we can merge https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/776221?

Nikerabbit added a comment.May 30 2022, 3:55 PM
In T306422#7967839, @dcausse wrote:

@abi_ can we consider this task done and close it so we can merge https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/776221?

Yes it can be reviewed and merged now. I am keeping this task open until the associated patches for workarounds and fixes we did on the fly during the upgrade are reviewed and merged.

gerritbot added a comment.Jun 1 2022, 3:33 AM

Change 793393 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] ttmserver-export: Add workaround for T258860

https://gerrit.wikimedia.org/r/793393

ReleaseTaggerBot added a project: MW-1.39-notes (1.39.0-wmf.15; 2022-06-06).Jun 1 2022, 4:00 AM
Maintenance_bot removed a project: Patch-For-Review.Jun 1 2022, 4:30 AM
Nikerabbit closed this task as Resolved.Jun 2 2022, 6:22 AM
• gonzalez.actor subscribed.Oct 21 2022, 3:42 PM
gerritbot added a comment.Nov 3 2022, 8:20 AM

Change 852735 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] Upgrade ElasticSearch version to 7.10.2

https://gerrit.wikimedia.org/r/852735

gerritbot added a project: Patch-For-Review.Nov 3 2022, 8:20 AM
Nikerabbit closed subtask T307998: Unable to run ttmserver-export.php: Container disabled! as Resolved.Dec 20 2022, 9:44 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL