Page MenuHomePhabricator
Create Task
Maniphest T308116

Update Elasticsearch to a new version for wikibase.dev/cloud
Closed, ResolvedPublic
Actions

Description

We are still running the unmaintained 6.54 version of Elasticsearch.

The 6.8.23 version is already available under the 6.8.23-wmde.6 tag, I've tested this out locally and it seems to work.

The 7.10 has plugins available but there is currently now new docker image packaged to use, it's also not clear if this is compatible with the chart we have.

AC

  • Decide on a version to use (6.8.23)
  • Elasticsearch is updated to a new version on wikibase.dev
  • Elasticsearch is updated to a new version on wikibase.cloud

Useful links

Related Objects

Event Timeline

• toan created this task.May 11 2022, 10:17 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 11 2022, 10:17 AM
• toan updated the task description. (Show Details)May 11 2022, 11:45 AM
• toan added a comment.EditedMay 11 2022, 1:37 PM

Ok i've tried it out a second time,

  1. setup 6.54 locally and re-indexed everything
  2. shutdown the Elasticsearch service (entered the pod and shut it off with the pid)
  3. Swap the new 6.8.23 image boots up with the following output
  4. Search seems to work with alias from old index etc.
kubectl logs elasticsearch-master-0 -f
[2022-05-11T13:21:34,261][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-master-0] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/ubuntu--vg-root)]], net usable_space [113.9gb], net total_space [467.5gb], types [ext4]
[2022-05-11T13:21:34,264][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-master-0] heap size [1gb], compressed ordinary object pointers [true]
[2022-05-11T13:21:34,367][INFO ][o.e.n.Node               ] [elasticsearch-master-0] node name [elasticsearch-master-0], node ID [4xHOckoeRBaXjcqBiz5H6g]
[2022-05-11T13:21:34,367][INFO ][o.e.n.Node               ] [elasticsearch-master-0] version[6.8.23], pid[1], build[default/docker/4f67856/2022-01-06T21:30:50.087716Z], OS[Linux/5.4.0-109-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2022-05-11T13:21:34,367][INFO ][o.e.n.Node               ] [elasticsearch-master-0] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.io.tmpdir=/tmp/elasticsearch-7834342987328626689, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Dlog4j2.formatMsgNoLookups=true, -Des.cgroups.hierarchy.override=/, -Xms1g, -Xmx1g, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker]
[2022-05-11T13:21:38,056][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [aggs-matrix-stats]
[2022-05-11T13:21:38,056][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [analysis-common]
[2022-05-11T13:21:38,056][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [ingest-common]
[2022-05-11T13:21:38,056][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [ingest-geoip]
[2022-05-11T13:21:38,056][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [ingest-user-agent]
[2022-05-11T13:21:38,056][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [lang-expression]
[2022-05-11T13:21:38,056][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [lang-mustache]
[2022-05-11T13:21:38,057][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [lang-painless]
[2022-05-11T13:21:38,057][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [mapper-extras]
[2022-05-11T13:21:38,057][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [parent-join]
[2022-05-11T13:21:38,057][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [percolator]
[2022-05-11T13:21:38,057][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [rank-eval]
[2022-05-11T13:21:38,057][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [reindex]
[2022-05-11T13:21:38,057][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [repository-url]
[2022-05-11T13:21:38,057][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [transport-netty4]
[2022-05-11T13:21:38,057][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [tribe]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-ccr]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-core]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-deprecation]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-graph]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-ilm]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-logstash]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-ml]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-monitoring]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-rollup]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-security]
[2022-05-11T13:21:38,058][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-sql]
[2022-05-11T13:21:38,059][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-upgrade]
[2022-05-11T13:21:38,059][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded module [x-pack-watcher]
[2022-05-11T13:21:38,059][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded plugin [experimental-highlighter]
[2022-05-11T13:21:38,059][INFO ][o.e.p.PluginsService     ] [elasticsearch-master-0] loaded plugin [extra]
[2022-05-11T13:21:45,228][INFO ][o.e.x.s.a.s.FileRolesStore] [elasticsearch-master-0] parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]
[2022-05-11T13:21:46,277][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [elasticsearch-master-0] [controller/80] [Main.cc@114] controller (64 bit): Version 6.8.23 (Build 31256deab94add) Copyright (c) 2022 Elasticsearch BV
[2022-05-11T13:22:30,590][INFO ][o.e.d.DiscoveryModule    ] [elasticsearch-master-0] using discovery type [zen] and host providers [settings]
[2022-05-11T13:22:32,757][INFO ][o.e.n.Node               ] [elasticsearch-master-0] initialized
[2022-05-11T13:22:32,757][INFO ][o.e.n.Node               ] [elasticsearch-master-0] starting ...
[2022-05-11T13:22:33,060][INFO ][o.e.t.TransportService   ] [elasticsearch-master-0] publish_address {172.17.0.25:9300}, bound_addresses {0.0.0.0:9300}
[2022-05-11T13:22:33,392][INFO ][o.e.b.BootstrapChecks    ] [elasticsearch-master-0] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-05-11T13:22:36,510][INFO ][o.e.c.s.MasterService    ] [elasticsearch-master-0] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {elasticsearch-master-0}{4xHOckoeRBaXjcqBiz5H6g}{AxS36FKuRkK3Sbt28S2YYw}{172.17.0.25}{172.17.0.25:9300}{ml.machine_memory=1610612736, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2022-05-11T13:22:36,515][INFO ][o.e.c.s.ClusterApplierService] [elasticsearch-master-0] new_master {elasticsearch-master-0}{4xHOckoeRBaXjcqBiz5H6g}{AxS36FKuRkK3Sbt28S2YYw}{172.17.0.25}{172.17.0.25:9300}{ml.machine_memory=1610612736, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {elasticsearch-master-0}{4xHOckoeRBaXjcqBiz5H6g}{AxS36FKuRkK3Sbt28S2YYw}{172.17.0.25}{172.17.0.25:9300}{ml.machine_memory=1610612736, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2022-05-11T13:22:36,552][INFO ][o.e.h.n.Netty4HttpServerTransport] [elasticsearch-master-0] publish_address {172.17.0.25:9200}, bound_addresses {0.0.0.0:9200}
[2022-05-11T13:22:36,553][INFO ][o.e.n.Node               ] [elasticsearch-master-0] started
[2022-05-11T13:23:22,575][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [elasticsearch-master-0] Failed to clear cache for realms [[]]
[2022-05-11T13:23:22,669][INFO ][o.e.l.LicenseService     ] [elasticsearch-master-0] license [825b4835-a834-4353-9943-e1117ca64a11] mode [basic] - valid
[2022-05-11T13:23:22,683][INFO ][o.e.g.GatewayService     ] [elasticsearch-master-0] recovered [10] indices into cluster_state
[2022-05-11T13:24:07,853][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-master-0] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[mwdb_69f10a19e0_content_first][0], [mw_cirrus_metastore_first][0]] ...]).
• toan moved this task from Migration backlog to Launch Migration Kanban (2022) on the Wikibase Cloud board.May 11 2022, 1:42 PM
• toan edited projects, added Wikibase Cloud (Launch Migration Kanban (2022)); removed Wikibase Cloud.
WMDE-leszek triaged this task as Medium priority.May 11 2022, 1:55 PM
Tarrow claimed this task.May 11 2022, 2:32 PM
Tarrow moved this task from Backlog to Doing on the Wikibase Cloud (Launch Migration Kanban (2022)) board.
Tarrow removed Tarrow as the assignee of this task.May 12 2022, 9:01 AM
Tarrow moved this task from Doing to Review on the Wikibase Cloud (Launch Migration Kanban (2022)) board.
Tarrow subscribed.

https://github.com/wmde/wbaas-deploy/pull/292

• toan claimed this task.May 12 2022, 9:22 AM
• toan updated the task description. (Show Details)May 12 2022, 10:30 AM
• toan removed • toan as the assignee of this task.May 12 2022, 10:48 AM
• toan updated the task description. (Show Details)

deployed to wikibase.dev

• toan mentioned this in T308115: Change default shard count for ElasticSearch on wikibase.dev/cloud.May 12 2022, 12:33 PM
• toan added a comment.May 12 2022, 12:35 PM

Production PR -> https://github.com/wmde/wbaas-deploy/pull/296

Deniz_WMDE moved this task from Review to Deploy To Production on the Wikibase Cloud (Launch Migration Kanban (2022)) board.May 12 2022, 12:45 PM
Tarrow moved this task from Deploy To Production to Done on the Wikibase Cloud (Launch Migration Kanban (2022)) board.May 13 2022, 8:32 AM
Tarrow closed this task as Resolved.Jun 23 2022, 9:14 PM
Tarrow claimed this task.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL