The elasticsearch backend is used by the Jenkins job mwext-browsertests-CirrusSearch-phantomjs. We set up a MediaWiki install that uses the elasticsearch local to the instance. It is not teared down / maintained afaik.

We can add a step to dish out the index on job completion. We will have to prevent the job from running concurrently on the same instance to avoid race condition.

@Manybubbles probably has clues :-]

@hashar: looks like the job you have mentioned is deleted.

One less job to take care about! I suspect the MediaWiki integration test of ElasticSearch rely on such a backend. Would have to look at its code.

Maybe the daemon keep track of connections being made to it, if there is none I guess we can safely remove it from the slaves.

This task is quite old now. Is this still an issue?

Yep. Here's a snapshot from integration-slave-trusty-1017 just now:

top - 21:28:42 up 72 days, 13:31,  1 user,  load average: 0.41, 0.42, 0.31
Tasks: 145 total,   1 running, 144 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.1 us,  0.2 sy,  0.0 ni, 99.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:   8176820 total,  7714436 used,   462384 free,   577756 buffers
KiB Swap:   499708 total,    20988 used,   478720 free.  3437668 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                                                
  973 elastic+  20   0 3521804 1.402g  27540 S   1.7 18.0 705:08.54 /usr/lib/jvm/java-7-openjdk-amd64//bin/java -Xms1G -Xmx1G -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSIni+
    1 root      20   0   29532   2680   1312 S   0.0  0.0  12:58.72 /sbin/init

In T89083#1856038, @Krinkle wrote:

Yep. Here's a snapshot from integration-slave-trusty-1017 just now:

Thanks. I'm going to bump priority up a bit so we take a look at this.

It doesn't look like we are using the phantomjs build. AFAIK it would be perfectly fine to turn off elasticsearch on these instances. In the future when we manage to move our browser tests out of labs and into the integration this will need to be revisited, likely standing up the elasticsearch instance at the beginning of a run and shutting it down after, if necessary.

In T89083#1862661, @EBernhardson wrote:

It doesn't look like we are using the phantomjs build. AFAIK it would be perfectly fine to turn off elasticsearch on these instances. In the future when we manage to move our browser tests out of labs and into the integration this will need to be revisited, likely standing up the elasticsearch instance at the beginning of a run and shutting it down after, if necessary.

So none of the unit tests run via Jenkins, and none of browser tests that run against a local build (if that exists) use ElasticSearch? (e.g. not talking about browser tests against beta labs or prod)

the unit tests run via jenkins, but they do not talk to elasticsearch. They are strictly unit tests. The browser tests don't run through integration, they run through a special instance in labs (cirrus-browser-bot.search.eqiad.wmflabs).

From what I can see, Discovery has resolved this on our end as we said that ElasticSearch can be turned off on the slaves. This is just waiting for someone from Release Engineering to actually do this.

Change 259301 had a related patch set uploaded (by Chad):
ci: remove elasticsearch from browsertest slaves

https://gerrit.wikimedia.org/r/259301

Change 259301 merged by ArielGlenn:
ci: remove elasticsearch from browsertest slaves

https://gerrit.wikimedia.org/r/259301