Page MenuHomePhabricator

Run Matrix trial using the modular.im-hosted instance
Closed, ResolvedPublic

Description

Real-time communication is somewhat of a pain point for the Wikimedia movement. There is a large contingent of IRC users, some of whom refuse to use proprietary or closed systems due to their poor alignment with movement values, and some of whom cannot use such systems because they depend on the openness of IRC orhave highly specialized workflows (various notifications, highlights, personal scripts, helper bots, vandalism tracking bots etc). Matrix / Riot.im has been raised as a potential alternative that has a modern user interface, preserves the openness of IRC (both in the sense of FLOSS and in having open groups) and supposedly has great bridging abilities (ie. can synchronize the conversation between a Matrix channel and an IRC/Slack/etc channel). How well it works in practice has yet to be seen however.

Thus, we need to run a to see if Matrix really is a good candidate for a chat system that's acceptable for most of the current IRC and Slack users. The trial should probably include:

  • inviting users of IRC (including IRCCloud) and Slack (and possibly other tools prominently used by the movement, like Google Chat or Telegram or Zulip, or the tool WMDE adopted (was that Mattermost?)) and usability experts to give it a test drive and see if they identify any major blockers
  • setting up bridges with IRC and Slack (and possibly other tools, as above) and see how well they work in practice
  • converting some IRC bots (at least some kind of logging bot and some kind of notifications / chat bot) to use Matrix and see if they work smoothly / what advantages we get
  • inviting teams to use Matrix as their primary communication channel for a trial period (3 months?)

See subtasks for blockers to staring the trial. (Nice to have but not a blocker: T230536)

After these are done, evaluate and make a go / no-go decision; if it's a no, try to turn the feedback into concrete blockers, otherwise we make a proposal to the executive team and if there is at least conditional support for Matrix as a canonical chat solution, ask ops for a self-hosted instance.

(Note: Matrix naming is a bit confusing. Matrix is the protocol, Riot is the most popular Matrix client, which we plan to use; Vector.im is the for-profit arm of the Matrix developer community, Modular.im is a SaaS (hosted Matrix server + Riot client) they operate, which we are using for the trial. Matrix and Riot are sametimes referred to as Matrix.org and Riot.im, after the domain name of the main website and the publicly available Matrix instance.)

See also:

Related Objects

Event Timeline

Tgr updated the task description. (Show Details)

Pinging some people who I think might be interested in this: @Chicocvenancio @Quiddity @kostajh @CKoerner_WMF @Milimetric @greg @Neilmatrix @CCicalese_WMF @MichaelSchoenitzer_WMDE @Samwilson @Johan @Qgil @Bmueller

The technical preparations for the trial are done; what's left is mostly discussion and documentation writing (see subtasks); help welcome!
The main questions to figure out: should the trial be open to the community (or WMDE, who I think have just adopted some other messaging system (Mattermost? Rocketchat?))? Should that mean non-WMF-staff can register to the Matrix instance (note we have a 50 user limit), or that they use other instances and we set up some federated channels?

I think having the trial open to non-WMF-staff might mean better future chances of adoption if we come to the conclusion that we like it, if it's not just limited to a specific subgroup of Wikimedians?

As an interested party, I'm curious about what's happening now as far as going ahead with the trial.

Welp, a month has passed so let me see if anyone is looking at this yet; I understand we all have a lot on our plates but it would be nice not to let this be entirely forgotten.

I got sucked into movement strategy and had no time to work on this at all; maybe I'll have a chance to wrap it up during the holidays. (Help welcome - see the subtasks for what needs to be done. They are not technical and don't require any kind of special skill. The service itself has been operational for quite some time.)

For what it's worth, if any hands are needed I can lend some.

@Tgr what help can people provide right now?

inviting users of IRC (including IRCCloud) and Slack (and possibly other tools prominently used by the movement, like Google Chat or Telegram or Zulip, or the tool WMDE adopted (was that Mattermost?)) and usability experts to give it a test drive and see if they identify any major blockers

Is this still a need? If so, I'll try it.

Also, I personally would like to try something different than Google Chat for Growth-Team and I think some others may be open to the idea too; maybe it would be possible to have that team use Matrix for a trial period?

I'm interested in helping out here.

I'll set up a trial for modular.im, the Matrix SaaS product, in the next couple of months and update this task when I do so.

@bcampbell that's good news! I'm reading New Vector's privacy policy and while they seem very nice and everything, they do host their VMs on Amazon's AWS (https://modular.im/terms-of-service#privacy-policy). Would this still be ok with the group of folks that want to try Matrix or do others feel strongly we should host our own servers?

I'm asking because I was just about to start writing docs for connecting to the instance that Gergo made. Either way I want to help; the recent realization that so many people seem to be on slack saddened me very much. We can't afford to be divided in this fundamental way. Brendan, if I can help set up the trial I'm happy to do that, and others if we want our own server, I'm more than willing to invest some real time here over the next few months.

@Milimetric For the sake of transparency and not creating more work for others, hosting our own Matrix server is most likely not going to happen. I'm speaking to the person at Mozilla who spearheaded their transition to Matrix this Friday and I'm looking forward to learning about how they made it happen.

I feel as though hosting our own instances is preferable, overall, in production. I understand for prototyping / buy-in using a hosted service is desirable, but the actual overhead for hosting is not that great. I get that volunteering other people's time isn't something that could be done, but I can volunteer a little of my SRE time to this prototyping phase in any case.

We don't run IRC ourselves. I don't know what the foundation's relationship with the freenode IRC network is, but paying someone to host Matrix for us should be no worse than our IRC situation is. Also no worse than Slack is.

The nice part of Matrix is that the protocol supports federated operation, which means it doesn't really matter now if we decide to host things ourselves or not: it's something we can fairly easily change our minds on later on. Even more easily if we can use our own domain for the "homeserver", which seems to be the component in Matrix where identities are hosted, rather than a domain controlled by the hosting company.

This is similar to the situation we have with email. Our email is hosted by Google, but we use the wikimedia.org domain for our addresses. I know we're not moving away from gmail, but if we did, the change would be mostly invisible to outsiders. We'd "just" change the domain to point at another mail server. (I know that in reality there would be more steps, but they could also be mostly invisible outside the organization.)

Hello, Neil from Matrix here.

It's worth noting that you already have a test Matrix instance provisioned for https://wikimedia.org it seems to have about 5 regular users though it is currently provisioned for up to 50. This is the instance that @Tgr referenced above. I'd be happy to help with any support on the Matrix/Modular side if that would help you to progress.

On the hosting/self hosting side if you ever want to migrate from modular we can give you a db dump and your signing keys at any point, so as @LarsWirzenius says you can migrate transparently whenever you like.

[Edited, initially I thought you had a modular specific server provisioned]

Thanks all for the offers to help! I think the immediate most useful thing would be to create a new user with Riot (the web, old/new iOS, old/new Android, desktop clients are all interesting targets; or if you feel strongly about using one of the alternative clients, that might be useful too), look at the basic things that can be done, take screenshots and write some documentation, as per T226631. (We have m:Matrix.org but that's very focused on using it as an IRC client, as opposed to chat client in general.) Then do the same with the internal matrix instance at wikimedia.riot.im, to the extent there are any differences (registration/login and privacy options, I guess).

I feel as though hosting our own instances is preferable, overall, in production. I understand for prototyping / buy-in using a hosted service is desirable, but the actual overhead for hosting is not that great.

I agree about self-hosting in general (although for people concerned about secrecy, Matrix can do end-to-end encryption for private rooms, in wich case hosting does not matter that much), but for the evaluation phase it seems like an unnecessary risk to me. For one thing, Matrix does not seem that easy to set up, there are half a dozen different services involved (homeserver, web UI, fileserver for sending attachments, identity service, integrations service...); for another, we'd want to be sure that any shortcomings we run into during evaluation are shortcomings of Matrix and mistakes we made when setting it up. Thirdly, something we run on Cloud Services infrastructure seems less trustable to me then a hosted solution, and I imagine we shouldn't install software in production without security review (which is again a commitment that shouldn't be made just to evaluate something).

I get that volunteering other people's time isn't something that could be done, but I can volunteer a little of my SRE time to this prototyping phase in any case.

Thanks for the offer! My personal opinion is that time would be better spent working on tools (such as seeing how hard it would be to port an existing IRC bot). That said, if you are interested with playing around with the server (I'm sure there are useful things to be learnt from it), we have a matrix project on Cloud VPS; let me know if you want to be added there.

Tgr renamed this task from Run Matrix trial using the Vector.im-hosted instance to Run Matrix trial using the modular.im-hosted instance.Jan 23 2020, 8:32 PM
Tgr updated the task description. (Show Details)
Tgr claimed this task.

The unofficial trial instance has been superseded by the official WMF-internal instance (wikimediafoundation.element.io, set up a year ago or so).

Some sort of open trial would still be useful (for legal and security reasons, the WMF instance cannot federate and can't connect to many things, at least for the moment), not necessarily via modular.im (now ems.element.io) though.