Page MenuHomePhabricator

Resolve current deprecation warnings in elasticsearch 5
Closed, ResolvedPublic

Description

Things deprecated in 5 are likely to be gone in 6. Various deprecation warnings are logged in production and should be resolved: https://logstash.wikimedia.org/goto/3ac74609059ac459827eac9752745bfc

Event Timeline

EBernhardson created this task.
EBernhardson removed a project: Epic.

Surprisingly some of these warnings are simply for very old indices. Somehow commonswiki_general is dated feb 2017, although we've certainly done full reindexes since then. It's possible reindex failed somewhere and we never noticed, the logs for a reindex are so large we don't actually check they all succeed.

eqiad elasticsearch cluster (not logstash) was out of sync with the apifeatureusage template in puppet causing it to create indices with deprecation warnings. I've updated the template from the one in puppet and new indices going forward should not log deprecation warnings. Some day we have to figure out how those templates get from logstash to elasticsearch, somehow or another it wasn't auto-magically deployed (should it be?).

Change 430250 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[operations/puppet@production] Convert text fields to string in apifeatureusage

https://gerrit.wikimedia.org/r/430250

Change 430251 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/extensions/CirrusSearch@master] Remove include_in_all mapping configuration

https://gerrit.wikimedia.org/r/430251

Change 430253 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/extensions/CirrusSearch@master] Pass string flags in query parameters

https://gerrit.wikimedia.org/r/430253

upgraded metastore on eqiad and codfw from 0.2 to 0.3 to fix more deprecation warnings about "index": "not_analyzed" which should be "index": "no". I'm not sure why but the minor upgrade didn't work so i forced a major (re-create and reindex) upgrade.

Change 430251 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Remove include_in_all mapping configuration

https://gerrit.wikimedia.org/r/430251

Change 430253 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Pass string flags in query parameters

https://gerrit.wikimedia.org/r/430253

I think i've gone through all the deprecation warnings from the last week and either resolved or submitted patches for them. There is one remaining that I haven't been able to track down:

Native scripts are deprecated. Use a custom ScriptEngine to write scripts in java.

This is only being reported by nodes in the codfw cluster.

Change 430496 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/extensions/CirrusSearch@master] Use string 'true' for boolean query param

https://gerrit.wikimedia.org/r/430496

Change 430496 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Use string 'true' for boolean query param

https://gerrit.wikimedia.org/r/430496

There are also some deprecation warnings coming from the phabricator index, I pinged them in T181393.

Moving to stalled/waiting, as we now need to wait a bit for updates to roll out and re-check for deprecation logs before declaring this done.

Change 430250 merged by Gehel:
[operations/puppet@production] Convert string fields to text in apifeatureusage

https://gerrit.wikimedia.org/r/430250

Rechecked the last weeks worth of logs, there are only 2 deprecation warnings both of which are acceptable. One is the phabricator index, it specifies an index property that has been removed in 6. It isn't a problem though, the index will continue to work in 6 and they are aware mappings need to be updated to work in 6. The other warning is related to our plugins, when a server starts it complains about using native scripts. This will be delt with when we upgrade the plugin to work with 6.