⚓ T272111 Elasticsearch, a CirrusSearch dependency, is switching to SSPL/Custom licence

Status	Assigned	Task
Open	None	T272111 Elasticsearch, a CirrusSearch dependency, is switching to SSPL/Custom licence
Resolved	lmata	T272238 Elasticsearch and Kibana are switching to non-OSI-approved SSPL licence
Resolved	bking	T280482 Validate that OpenSearch is a viable replacement for Elasticsearch for CirrusSearch

Gacelperfinian created this task.Jan 15 2021, 1:21 AM

Restricted Application added a project: Discovery-Search. · View Herald TranscriptJan 15 2021, 1:21 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Gacelperfinian renamed this task from Elasticsearch will switch to SSPL to Elasticsearch, a CirrusSearch dependency, is switching to SSPL/Custom licence.Jan 15 2021, 1:22 AM

Gacelperfinian added a project: Software-Licensing.Jan 15 2021, 1:24 AM

Gacelperfinian updated the task description. (Show Details)

Gacelperfinian moved this task from Backlog to License switches on the Software-Licensing board.

WIll CirrusSearch rely on ElasticSearch in the near future (say on version 7.x)?

Yes. CirrusSearch is currently on 6.5.4. It will likely upgrade to 6.8.x soon. These versions are EOL and stopped receiving security updates from elastic in Nov 2020.

Will CirrusSearch rely on ElasticSearch in long-term?

Yes? No? Maybe? It will take months to figure this out.

According to task T213996, the switch from MongoDB had happened because it was removed in Debian and therefore will be unsuitable for long-term use., and therefore it was unclear if a precedent was achieved. Are there any policy change or clarification if another dependency announced to switch to a license that may not be suitable for MediaWiki?

The rule of thumb, it may or may not be a strict rule somewhere, is that WMF deploy's software with OSI approved licenses. There is at least some mention under development policys that new services must be licensed under an OSI-approved license.

Also for notetaking: Kibana will also be affected by this change, but I am not sure if WMF stack uses Kibana in some way (if there is any, this should be on another task anyways).

The Translate extension also uses ElasticSearch.

dcausse subscribed.Jan 15 2021, 8:46 AM

Nemo_bis subscribed.Jan 15 2021, 8:47 AM

fgiunchedi subscribed.Jan 15 2021, 10:48 AM

hashar updated the task description. (Show Details)Jan 15 2021, 10:48 AM

• MoritzMuehlenhoff subscribed.Jan 15 2021, 10:50 AM

bd808 subscribed.Jan 15 2021, 5:01 PM

• dpifke subscribed.Jan 15 2021, 7:03 PM

Jdforrester-WMF subscribed.Jan 16 2021, 9:42 PM

• taavi subscribed.Jan 17 2021, 11:47 AM

I filed T272238: Elasticsearch and Kibana are switching to non-OSI-approved SSPL licence for WMF's ELK setup

Tgr subscribed.Jan 18 2021, 5:26 AM

Gehel moved this task from needs triage to Current work on the Discovery-Search board.Jan 18 2021, 4:03 PM

Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.

Base subscribed.Jan 19 2021, 4:20 PM

• Mholloway subscribed.Jan 19 2021, 8:22 PM

revi subscribed.Jan 20 2021, 3:16 PM

colewhite subscribed.Jan 20 2021, 8:04 PM

AntiCompositeNumber subscribed.Jan 20 2021, 10:51 PM

WMDE-leszek subscribed.Jan 21 2021, 6:05 AM

akosiaris subscribed.Jan 21 2021, 9:44 AM

akosiaris added a subtask: T272238: Elasticsearch and Kibana are switching to non-OSI-approved SSPL licence.Jan 21 2021, 9:54 AM

jmnote subscribed.Jan 21 2021, 10:10 AM

Ladsgroup subscribed.Jan 22 2021, 4:49 AM

Kronoss subscribed.Jan 22 2021, 10:02 AM

Nemo_bis renamed this task from Elasticsearch, a CirrusSearch dependency, is switching to SSPL/Custom licence to Elasticsearch is going proprietary with a SSPL/custom licence; forks coming.Jan 24 2021, 4:47 PM

Nemo_bis added projects: Wikimedia-Logstash, MediaWiki-extensions-Translate, WMF-General-or-Unknown.

Nemo_bis updated the task description. (Show Details)Jan 24 2021, 4:52 PM

This isn't the task for the Logstash or Translate issues. Please see T272238: Elasticsearch and Kibana are switching to non-OSI-approved SSPL licence for the former and create a new task for the latter, as implied above.

Jdforrester-WMF renamed this task from Elasticsearch is going proprietary with a SSPL/custom licence; forks coming to Elasticsearch, a CirrusSearch dependency, is switching to SSPL/Custom licence.Jan 24 2021, 6:11 PM

CBogen moved this task from Incoming to Blocked/Waiting on the Discovery-Search (Current work) board.Jan 25 2021, 4:52 PM

Bugreporter mentioned this in T250227: Investigate and evaluate hCaptcha to replace Wikimedia's Fancy Captcha.Jan 25 2021, 7:29 PM

• MPhamWMF subscribed.Feb 1 2021, 4:31 PM

Jdforrester-WMF mentioned this in T271777: Bump rufin/elastica (and related libraries) to versions that support PHP 8.0.Feb 12 2021, 12:30 AM

Framawiki subscribed.Feb 13 2021, 8:23 PM

Addshore subscribed.Feb 22 2021, 10:14 AM

freephile subscribed.Feb 24 2021, 2:42 PM

RheingoldRiver subscribed.Feb 24 2021, 3:59 PM

Bryandamon subscribed.Feb 24 2021, 8:42 PM

Krenair subscribed.Apr 13 2021, 7:56 AM

Gehel added a subtask: T280482: Validate that OpenSearch is a viable replacement for Elasticsearch for CirrusSearch.Apr 19 2021, 8:21 AM

Lens0021 subscribed.Jun 18 2021, 4:29 PM

herron mentioned this in T225125: Migrate Elasticsearch from deprecated Gelf logstash input to rsyslog Kafka logging pipeline.Aug 30 2021, 6:41 PM

• toan mentioned this in T297002: Elasticsearch 6.5.4 docker image contains critical security issues.Dec 9 2021, 11:09 AM

lmata closed subtask T272238: Elasticsearch and Kibana are switching to non-OSI-approved SSPL licence as Resolved.Feb 14 2022, 4:16 AM

For the moment, we're staying on Elasticsearch 7.x (once we've migrated from 6.x to 7.x), but we're blocked on upgrading past 7.10.2 because of licensing. There is a larger question of what exactly are the needs in term of Search that needs to be resolved.

There isn't a clear acceptance criteria on this task, so closing it for now. Feel free to re-open as needed.

In T272111#7730031, @Gehel wrote:

For the moment, we're staying on Elasticsearch 7.x (once we've migrated from 6.x to 7.x), but we're blocked on upgrading past 7.10.2 because of licensing. There is a larger question of what exactly are the needs in term of Search that needs to be resolved.

There isn't a clear acceptance criteria on this task, so closing it for now. Feel free to re-open as needed.

I don't think staying on Elasticsearch 7.x is a long-term solution, is it? I think we can consider this ticket resolved once CirrusSearch has moved to using something that is properly open source under an OSI license.

Gehel edited projects, added Discovery-Search; removed Discovery-Search (Current work).Mar 4 2022, 7:45 AM

Gehel moved this task from needs triage to elastic / cirrus on the Discovery-Search board.

@Legoktm, see T272238 and T280482 - it seems OpenSearch is the relevant replacement in the future, and this task can be left closed.

RhinosF1 subscribed.Apr 10 2022, 4:45 PM

Aklapper mentioned this in T316990: Cirrussearch -> Elasticsearch licensing issue.Sep 4 2022, 10:16 AM

dcausse mentioned this in T317284: The search no longer works with the WMF28 (CirrusSearch).Sep 8 2022, 9:22 AM

TJones mentioned this in T317476: Filter and sort search results of Japanese kana search queries in accordance with how much of the query appears as a consecutive substring.Sep 12 2022, 6:38 PM

TheresNoTime removed a subscriber: RhinosF1.Dec 15 2022, 11:36 PM

Ciencia_Al_Poder subscribed.Dec 23 2022, 5:14 PM

TJones mentioned this in T330893: Map Romanian s&t with comma to cedilla internally.Mar 1 2023, 8:44 PM

Strainu subscribed.Mar 2 2023, 11:42 AM

bking subscribed.Feb 26 2024, 10:53 PM