Page MenuHomePhabricator

Evaluate Matrix / Riot.im
Open, MediumPublic

Description

Context

Real-time communication is somewhat of a pain point for the Wikimedia movement. There is a large contingent of IRC users with highly specialized workflows (various notifications, highlights, personal scripts, helper bots, vandalism tracking bots etc) for whom moving to a different chat system is probably a no-go. There is also a large contingent of less technical users for whom the cost of learning IRC (with all that setting up a convenient environment involves - cloaking, a bouncer, notifications etc) is prohibitive. Using a different system for them creates a rift in the community, and is often contentious (as the most popular options are not free software, often not privacy-friendly, and don't support open community use cases well). Matrix / Riot.im (backend and default client for the same chat system) has the promise of fixing this problem - it is backwards-compatible with IRC on a low level and aims to provide a modern UI with all the bells and whistles people have come to expect from a chat system.

We should see what it would take for Matrix to be the offical chat system recommended for Wikimedia community members (IRC users could stay on IRC if they wanted, the two systems are fully interoperable) and see if we can help them get there. As a first step we should evaluate whether there are any features / UI improvements we'd want and whether those are must-haves or nice-to-haves.

Wikimedia use cases

Evaluation

Overview

Matrix is an open messaging prootocol using a federated network of servers (called homeservers). Users and rooms are scoped to homeservers (with rooms being able to span many homeservers via aliases). Communication happens by clients talking to their own homeserver which is responsible for authenticating them, and homeservers synchronizing the messages with all the other homeservers who have clients participating in that conversation, ensuring eventual consistency. (Internally, it's more of a data synchronization protocol, with chat histories, not messages, being the first-class citizens; the "How does it work" section of matrix.org has a fairly accessible explanation.) There are "official" clients for all major platforms (web, iOS, Android, Win/Mac/Linux desktop), and a large variety of third-party clients (and also a few third-party server implementations). UX-wise, the official clients aim for a Slack-like experience (and the protocol is also in some ways a reimagination of the popular messaging features, like message editing or emoji reactions, for a federated world).

Aside from its federated nature, the distinguishing features compared to other chat applications are the ability to do end-to-end encryption (on top of the normal, piece-wise HTTPS encryption and signing between client and homeserver, and homeserver and homeserver) for group chat, and a focus on seamlessly bridging with other communication networks.

Features

The official server and clients offer a similar feature set to Slack and its competitors: permanent identities with constant presence and notifications; searchable chat histories; rich text including link previews and attachments; avatars, stickers, emoji responses and bot integrations; channel previews; typing notifications and read receipts; message editing and removal. The one major Slack feature missing is threading (#2349; planned for this year).

The look-and-feel is inspired by Slack (although somewhat less polished - UX was developer-driven until a year ago, when they could afford hiring a design team, and they are still playing catchup). See https://about.riot.im/features for some screenshots.

There's also support for integrated VoIP calling and audio / video conferencing, and embedding widgets, but UX-wise those are more on the experimental end.

One of the strengths of Matrix is compatibility; there's a large variety of bridges to other communication networks.

From a developer point of view, the protocol is extensible and is by default based on JSON REST APIs. (see docs/sandbox) There's a number of client libraries / SDKs around

Privacy, safety and security

Messages are sent to all homeservers participating in the discussion (i.e. when a message is sent to a chatroom, it is first sent to the user's homeserver, and then that homeserver distributes it to the homeservers of all the other users who are in the room). It is possible to delete messages once they have been sent (by the author, or by a moderator), and to configure message rentention policies (per-room and per-server time limits after which messages are automatically deleted); of course enforcement depends on the cooperation of all involved servers and clients. Beyond the usual privacy options (private rooms, invitation-only rooms, per-room options for room previews and availability of chat history before one's joining) Matrix also allows homeserver-only rooms and end-to-end encrypted rooms. When all users are from the same homeserver (whether it is declared as homeserver-only or not), no information leaves the homeserver. With E2E encryption, the homeservers cannot read the messages, but it does not protect against metadata collection and traffic shape analyis (who talked to whom when, message length etc). That makes Matrix the only non-fringe chat system today with E2E encrypted group chat. The related UX, such as device cross-signing, is currently somewhat substandard, but planned to be a major focus for the next few months.

When using matrix.org or modular.im, the data is hosted in Europe, on UpCloud or AWS servers. For data not under the user's or homeserver's control (such as operational logs), the retention period is 180 days. (full privacy policy)

Moderation is fairly advanced compared to other chat systems (this was cited as a decisive reason for Mozilla adopting Matrix). Room moderators can delete messages, kick or ban users and set server ACLs (such as banlists); room admins can also change configuration (e.g. enable integrations). Server admins can send global notices, delete/erase users, delete rooms and groups, hide room directory entries and delete room aliases, reset passwords. Users can flag messages and have personal ignore lists. Due to the federated nature, there is no IP-based enforcement as the IP of a user is only known to their homeserver.

Matrix itself has not been security audited, but the French state agency allegedly audited their fork. There have been two major security incidents, neither strictly related to the Matrix network itself: a matrix.org server breach in April 2019 (via weakly protected CI servers), and a critical flaw in the French government's system (reportedly specific to their auth integration). There were four security releases last year (1, 2, 3, 4; two of them labeled critical). The encrytpion library went through an external audit.

Governance and funding

Matrix is an open protocol owned by a non-profit, managed via open governance process, evolved through an RfC-style process. The official server and client implementations are open-source (Apache 2.0); from personal experience the developer team is very responsive, both on their issue trackers and in chat. The company doing the development initially relied on crowdfunding, then went through two rounds of VC funding ($5M in 2018 January from secure messaging company Status.im and $8.5M in 2019 November from multiple SaaS funders); it now has a SaaS offering (modular.im and work with the French government and German army who plan on using Matrix as the foundation of their communications systems.

(For comparison with the other large FLOSS Slack competitors, Mattermost raised around $40M in total, and Rocket.Chat $5M, according to Crunchbase. Slack and Telegram are between $1-2B.)

Adoption

Major Big Open organizations that have adopted Matrix include Mozilla, KDE and Purism.

There are about a dozen mainstream alternative clients, and many fringe ones, plus three alternative server implementations.

The federated part of the Matrix network is estimated to have about 3.5M user accounts.

The French government and German army plan to use a fork of Matrix for their internal communications infrastructure.

Roadmap

Some of the 2020 priority focus areas for the Matrix / Riot team include end-to-end encrypted rooms, UX for non-technical users, group management, and bridges (connecting to other chat systems). See the 2019 report / 2020 roadmap for more details.

Event Timeline

Tgr triaged this task as Medium priority.Jan 30 2018, 10:05 PM
Tgr created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 30 2018, 10:05 PM
Tgr updated the task description. (Show Details)Jan 30 2018, 10:06 PM

jfyi @GWicke has been an early adopter of Matrix.

Qgil moved this task from To triage to Team radar on the Developer-Advocacy board.
jrbs added a subscriber: jrbs.Feb 1 2018, 6:39 PM
Johan added a subscriber: Johan.Feb 28 2018, 6:42 AM
This comment was removed by Johan.
Johan added a comment.Feb 28 2018, 6:47 AM

@Tgr Do you feel like setting this up?

Tgr added a comment.EditedFeb 28 2018, 7:04 AM

Do we want a hosted version? We don't seem to have a problem with using Freenode's servers, and this would be roughly comparable.

Johan added a comment.Feb 28 2018, 7:08 AM

I had just assumed that we would, but if we don't, then everything gets easier, of course.

Tgr added a comment.Feb 28 2018, 8:53 AM

At the very least for evaluating the usability, the official network should be fine IMO. It's bridged with Freenode, so not too dissimilar with how a local setup would work.

At the very least for evaluating the usability, the official network should be fine IMO

(Regarding usability, also see my remarks on https://meta.wikimedia.org/wiki/Talk:Matrix.org when testing a few weeks ago.)

Joe added a subscriber: Joe.Mar 16 2018, 12:00 AM

I'm going to ping @cwdent as he helped set up a test for fr-tech. He might have feedback here.

At the very least for evaluating the usability, the official network should be fine IMO. It's bridged with Freenode, so not too dissimilar with how a local setup would work.

One of the complaints I've heard about IRC is lack of truly private channels (besides 1:1 OTR) and while this may be a misconception regarding Slack it does seem like a reasonable thing to want. Hosting our own is an unknown quantity of work but it seems achievable if WMF decides to put resources into it. Not to put the cart before the horse, I agree that the public servers are fine for at least evaluation.

I used matrix/riot quite a bit when Yuvi was into it and set up a server but it was buggy and undocumented at the time. I'll be very interested to take another look.

FWIW I think one of the biggest things we stand to lose is the Freenode community/ecosystem so I hope whatever we do can bridge the gap effectively. The IRC bot for Riot.im was pretty bad at the time (cwd[m] was ghosting for like 6 months after I shut down the server)

One of the complaints I've heard about IRC is lack of truly private channels (besides 1:1 OTR)

Agreed. There are (at least) 2 aspects to this:

  • It is fairly hard to set-up private group chats that are permanent, e.g. for remote team-mates who want somewhere to socialize off the record. -- Hard because we cannot just re-use an existing channel's white-list/exemption-list (afaik).
  • It is fairly technical/cumbersome to set up temporary small group chats, e.g. 3 or 4 colleagues wanting to talk about something specific and private/sensitive, at length. -- Technical because it requires knowing that you can type the commands "/j ##examplechannelfoobarbang" at any time, and then dealing with the options of either using "/invite $user" or manually explaining to people how to join your new channel.

and while this may be a misconception regarding Slack

I think you might be referring to this recent news: https://thenextweb.com/apps/2018/03/22/slacks-new-policy-lets-bosses-read-employees-dms-without-consent/

Tgr added a comment.Mar 29 2018, 7:49 PM

Matrix supports end-to-end encryption these days so there is no need to host our own instance for truly private channels. No idea how the user experience is for E2E encryption, though.

IRC bridging is definitely the main reason we are looking at Matrix (as AFAIK no other software even has that option). I think it works pretty well these days, only used it a few times though.

Tgr added a comment.Mar 30 2018, 10:16 AM

If anyone is interested in testing the Matrix-IRC bridge, Mark has set one up between #mediawiki and #mediawiki-general:matrix.org.

Tgr added a comment.Apr 7 2018, 11:07 AM

We are collecting issues in #mwstake-matrix:matrix.org (and the Matrix ppl have kindly triaged them and explained the ones that were misunderstandings).

The IRC bridge is pretty decent in my experience; there are some small annoyances (such as long usernames getting truncated) and it's a bit laggy (which we could probably solve by running our own instance) but I'd definitely recommend it over normal web IRC interfaces like webchat.freenode.net. You need to register first so it's maybe no replacement to the "click here to join" IRC links we currently have (although if we run our own instance we could provide one-click login with Wikimedia SSO).

Clicking the link toward https://riot.im/app/#/room/#mediawiki-general:matrix.org respond me that the channel doesn't exists. Maybe I need some additional configuration though. I already accepted a spontaneous request by Freenode to join what I guess was a bridge, maybe on my laptop client (Fractal), but I guess that anyway it's synchronized in the browser client and it's not the problem.

Tgr added a comment.Aug 29 2018, 11:14 AM

Clicking the link toward https://riot.im/app/#/room/#mediawiki-general:matrix.org respond me that the channel doesn't exists.

Sorry, not sure where I got that from. The room is #mediawiki:matrix.org.

Tgr moved this task from Backlog to Next on the User-Tgr board.Feb 23 2019, 7:16 AM
grin added a subscriber: grin.Mar 14 2019, 3:18 PM
Arkanosis added a subscriber: Arkanosis.
Tgr updated the task description. (Show Details)May 6 2019, 3:57 AM
813gan added a subscriber: 813gan.EditedMay 15 2019, 6:16 PM

Matrix supports end-to-end encryption these days so there is no need to host our own instance for truly private channels. No idea how the user experience is for E2E encryption, though.

E2E is in late beta, there are some minor issues with key management if you are clearing cookies/storage.
Keep in mind that E2E works only inside Matrix. Bridge have to be able to decrypt communication to be usable.

Also, matrix.org exist only to help community grow and will not be running forever.

IRC bridge is nearly transparent. Both private and group chat works without client-side configuration.
General user experience of Riot is, in my opinion, very good both on desktop-web and mobile.

JJMC89 added a subscriber: JJMC89.May 16 2019, 6:40 AM
greg added a subscriber: greg.May 19 2019, 9:40 AM
Tgr mentioned this in Matrix.May 19 2019, 1:43 PM
Tgr added a comment.May 20 2019, 2:06 PM

TL;DR of the Matrix moderation guide:

  • room moderation:
    • three power levels by default (normal, moderator, admin)
      • moderators can delete messages (also all users can self-delete), kick or ban users, and set server ACLs (currently only available via developer toolbar)
      • admins control channel behavior (enable integrations etc)
      • users can promote others to a lesser or equal power level than their own, and demote others who have a lesser power level than their own
    • kick/ban targeting is limited to Matrix IDs (no IP bans as IP is only known to the user's homeserver) and entire homeservers (server ACL) -> this seems to imply you can't have sane moderation if you allow guest accounts
    • bridges typically do not transfer moderation actions (exceptions exist, e.g. kicks are transferred to/from IRC, and IRC bans are honored on Matrix (but not the other way around))
  • server moderation
    • server admins can send global notices, delete/erase users, delete rooms and groups, hide room directory entries and delete room aliases, reset passwords (not really clear which of these apply to users/rooms local to the homeserver, and which to all)
    • server admins can remove content (not clear what that means for users whose client has already synced that content)
    • server admins can see the IP of local users (there is no IP banning though, that would have to be done on the network level)
    • there's no GUI for most of these
  • users can flag/report content (seems like this is available on the Riot phone apps but not Riot web?)
  • users can add other users to an ignore list
  • the entire Matrix protocol is a huge JSON REST API, so creating tooling for moderation is easy
grin added a comment.May 20 2019, 3:47 PM
  1. If one's running a real room then "power levels" ought to be used a bit differently to prevent problems later: one shall use "admin" as level 90, and "owner" as level 100, and there may be used a level between admin and normal to let people to invite others or to change the topic of the room. The main reason is that equivalent level users cannot change one another, so without the admin+ level admins cannot be demoted, either by request ("I have lost the client keys please remove me") or due to loss of trust.
  1. Bridges are only good to connect the text body chat part; generally no special features get through without problems, including encryption, room and user state changes, and various "rich" content posts.
  1. Server admin actions are all local. They can fiddle with local users, local public posts (unless encrypted) and server connections. They can "remove" content by forcibly *redact* a locally generated message, which will be redacted on other servers (which in fact causes the message to be replaced by a redaction status, so gateways which transfer in real time have no way to do it later). It only have a real meaning in the room history. Server admin UI is in the works, until then it's done by REST calls.
  • Full history is kept for everything. ...unless the homeserver admin decides to purge otherwise. Since it's SQL there's also useable searching available. I found it pretty convenient to find and retrieve posted content.
  • Encryption is "proper", so right now it requires full mutual verification of device keys for all the participants.
  • Encrypion is also secure so
    • one cannot read messages created before joining,
    • one cannot read messages with session keys one's lost,
    • which means that care should be taken to keep the keys safe and secure.
  • Encryption generally works well, but I haven't seen large rooms using it yet.
  • Right now homeserver is python-based synapse, which is not pretty optimal but easy for rapid development. It can use workers to spread parts of the system on several servers, but generally suffers from one-threadedness. Some server implementations are in the works, written in C++ or Rust, but they're not ready yet. Db is psql.
  • There are lots of protocol development in the background (with no breaking changes in the majority case), like "groups", "message feedback" ("like"), and various security related stuff. It's handled on github as "requests for comments" (MSCs, https://github.com/matrix-org/matrix-doc/pulls?q=is%3Apr+is%3Aopen+label%3Aproposal)
Kelson added a subscriber: Kelson.Aug 1 2019, 4:44 PM
Tgr updated the task description. (Show Details)Aug 15 2019, 9:59 AM

Moved the trial plans (still not clear if that will be a WMF-only or WMF-mostly trial) to T230531: Run Matrix trial using the modular.im-hosted instance.

crusnov added a subscriber: crusnov.Jan 8 2020, 5:59 PM
Tgr updated the task description. (Show Details)Jan 19 2020, 12:59 AM
Tgr updated the task description. (Show Details)Jan 19 2020, 4:49 AM
Tgr updated the task description. (Show Details)Jan 19 2020, 7:24 AM
Tgr updated the task description. (Show Details)Jan 19 2020, 7:33 AM
Tgr updated the task description. (Show Details)Jan 19 2020, 8:41 AM
Tgr updated the task description. (Show Details)Jan 19 2020, 11:49 PM
Tgr updated the task description. (Show Details)Jan 20 2020, 12:30 AM