Page MenuHomePhabricator

Vagrant role provisions wrong elasticsearch version
Closed, ResolvedPublic

Description

After pulling up a new VM and enabling the cirrussearch role, the maintenance scripts on all wikis fail with an error message that says Elasticsearch 2.x is expected and 1.7.3 is installed.

Event Timeline

Tgr created this task.Jun 24 2016, 3:46 PM
Restricted Application added projects: Discovery, Discovery-Search. · View Herald TranscriptJun 24 2016, 3:46 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald Transcript
EBernhardson added a subscriber: EBernhardson.

the problem is we havn't been able to put the new version of elasticsearch into wikimedia apt repositories. It doesn't support multiple versions of the same package and the old version is still being used in production. As temporary workaround you can grab the elasticsearch package from elastic.co and use sudo dpkg -i elasticsearch-2.3.3.deb

I think we still have a custom apt repo for MediaWiki-Vagrant somewhere that could be used to provide the newer package version to MW-V. I know @yuvipanda set one up for us at some point, but I don't exactly remember if we have ever used it.

bd808 added a comment.Jul 9 2016, 9:32 PM

Caused by rMWVA8371db5fb261: Upgrade to elastic 2.3.3 which forced the version to a version not available in MediaWiki's apt repo. That patch probably should not have been merged. Hopefully we are just a few days away from resolving T136001: [EPIC] Upgrade elasticsearch cluster supporting logging to 2.3 which should unblock updating the apt repo.

debt triaged this task as High priority.Jul 12 2016, 10:20 PM
debt added a subscriber: debt.

moving to higher priority - after the elasticsearch upgrade happens, this can be done.

that patch was merged because CirrusSearch requires elasticsearch 2 now, so it was broken either way. Either it fails due to the elasticsearch version not being accepted by the cirrus maintenance scripts, or it fails because it tries to install an unavailable elasticsearch .deb, either way its broken and this is closer to being fixed.

getting closer, but we need to perform a release of our elasticsearch modules against 2.3.4. After that will be able to close this out as fixed.

debt assigned this task to dcausse.Aug 2 2016, 6:49 PM
debt edited projects, added Discovery-Search (Current work); removed Discovery-Search.

Change 302699 had a related patch set uploaded (by DCausse):
Upgrade to elastic 2.3.4

https://gerrit.wikimedia.org/r/302699

Change 302699 merged by jenkins-bot:
Upgrade to elastic 2.3.4

https://gerrit.wikimedia.org/r/302699

debt added a comment.Aug 16 2016, 10:08 PM

Hey, @EBernhardson can you verify this, please?

EBernhardson closed this task as Resolved.Aug 17 2016, 6:20 PM

built a new vagrant box from scratch, verified works.

Tgr reopened this task as Open.Dec 8 2016, 7:41 AM

This now breaks any time the wikimedia apt repo and the vagrant version are not in sync. E.g. right now the apt repo is at 2.4.2 so when provisioning I get

==> default: Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold --force-yes install elasticsearch=2.3.5' returned 100: Reading package lists...
==> default: Building dependency tree...
==> default: Reading state information...
==> default: E: Version '2.3.5' for 'elasticsearch' was not found

Is there really a need for the vagrant role to specify the version number?

I'm not aware that we upgraded to 2.4.2, we haven't released our elasticsearch plugins for 2.4.2.
The reason we need to specify the version number here is that the script to install plugins requires the version number. And I did not find a way to extract this information from puppet without specifying the version value manually.

Change 325908 had a related patch set uploaded (by Gergő Tisza):
Update elasticsearch version to match Wikimedia apt repo

https://gerrit.wikimedia.org/r/325908

Tgr added a comment.EditedDec 8 2016, 10:15 AM

The patch above makes provisioning work again for me. My apt knowledge is very limited but https://apt.wikimedia.org/wikimedia/dists/trusty-wikimedia/thirdparty/binary-amd64/Packages seems to be the thing to look at, and that refers to 2.4.2.

If you need the version number in puppet, maybe you can set it to ensure=latest, and write a facter check for what that is (I think it can be looked up via apt-cache policy)?

@Tgr thanks and sorry about that, we are looking into fixing the repo.
Concerning puppet and ensure=latest I'll have another look and try to fix it.

Change 325931 had a related patch set uploaded (by Faidon Liambotis):
aptrepo: pin ElasticSearch version

https://gerrit.wikimedia.org/r/325931

Deskana closed this task as Resolved.Dec 8 2016, 6:52 PM

Change 325908 merged by jenkins-bot:
Update elasticsearch version to match Wikimedia apt repo

https://gerrit.wikimedia.org/r/325908

Change 325931 merged by Alexandros Kosiaris:
aptrepo: pin ElasticSearch version

https://gerrit.wikimedia.org/r/325931