Page MenuHomePhabricator

Elasticsearch and Kibana are switching to non-OSI-approved SSPL licence
Open, MediumPublic

Description

Per https://www.elastic.co/blog/licensing-change Elasticsearch (which stores data collected by Logstash) and Kibana (which displays log data) will be switching to the Server Side Public License which according to Wikipedia is not OSI approved nor Debian Free Software Guidelines compatible.

Starting with the upcoming Elastic 7.11 release, we will be moving the Apache 2.0-licensed code of Elasticsearch and Kibana to be dual licensed under SSPL and the Elastic License, giving users the choice of which license to apply.

see also T272111: Elasticsearch, a CirrusSearch dependency, is switching to SSPL/Custom licence for use of Elasticsearch in MediaWiki extensions

Event Timeline

jcrespo added a subscriber: jcrespo.

Adding observability team, although for initial reactions not sure if a separate ticket for infrastructure logs vs search code is the right approach...

This may affect plans to use the Elastic Common Schema for logging.

https://github.com/elastic/ecs still appears to be Apache 2.0 licensed. (But https://github.com/elastic/elasticsearch hasn't been updated either, so maybe a change is pending.) If we use ECS and its license changes to SSPL, that may create headaches for downstream users of MediaWiki.

(Legal can chime in, but copyrightability of a logging schema—and thus whether or not the license even matters—is probably directly dependent on the outcome of https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_Inc.)

Confluent did the same thing a few years ago: https://www.confluent.io/blog/license-changes-confluent-platform/
This has prevented us from using some of their free products, even though they were the best solutions.

I 've marked T272111 as a parent of this task for greater visibility. This one seems more generic than the specific one to CirrusSearch hence this relationship, but feel free to undo.

Amazon has announced plans to fork Elasticsearch and Kibana under the original Apache 2.0 license (starting from ES 7.10)
https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/

Apparently there is a more community-oriented fork in the works, too: https://logz.io/blog/open-source-elasticsearch-doubling-down/

I 've marked T272111 as a parent of this task for greater visibility. This one seems more generic than the specific one to CirrusSearch hence this relationship, but feel free to undo.

Can't we just mark this as dupe? It's pretty clear there will be a viable fork, so it will "only" be a matter of picking a new upstream (or possibly joining the formation of one). Unless we end up with multiple incompatible upstreams (seems unlikely in the short term), I doubt we'd pick different ones for the various pieces of software.

I 've marked T272111 as a parent of this task for greater visibility. This one seems more generic than the specific one to CirrusSearch hence this relationship, but feel free to undo.

Can't we just mark this as dupe? It's pretty clear there will be a viable fork, so it will "only" be a matter of picking a new upstream (or possibly joining the formation of one). Unless we end up with multiple incompatible upstreams (seems unlikely in the short term), I doubt we'd pick different ones for the various pieces of software.

While that scenario may indeed (and I actually hope) pan out, assuming that it will without input from the respective teams would be presumptuous of me. I 'd much rather each interested party/team became aware of the issue, the potential solutions, benefits as well as concerns (pooling efforts and adopting the same solution, whatever that may be, is a very valid concern in my book and I guess yours as well) and came to a decision weighing all of those instead of just assuming from the beginning that 1 solution will be adequate for everyone.

Legoktm triaged this task as Medium priority.Jan 25 2021, 7:30 PM
Legoktm added a subscriber: Legoktm.

Have they announced when the 7.11 release is happening? It's not clear to me how long we have to figure out what to do next.

We have as long as we want to figure out what to do next, I don't think the day they drop 7.11 changes anything. CirrusSearch doesn't even support 7.x today, other uses at WMF may differ.

Elastic does not announce release dates in advance.

CirrusSearch is still on ES 6, which will become EOL on 2020-11-20.
Production logging is on ELK 7.10 as of two weeks ago (T234854). It will become EOL on 2022-05-11, but won't be maintained after the 7.11 release.

It appears that the two groups that announced forks (AWS and logz.io) may end up are working together on the same fork (per https://logz.io/blog/truly-doubling-down-on-open-source-2/, https://twitter.com/horovits/status/1353780274686001152). Both have said that more details are coming "in the next few weeks".