Page MenuHomePhabricator

Deploy a self-hosted public Matrix server instance
Open, Needs TriagePublic

Description

Draft Requirements
(provided by ERC)

  • Publicly accessible: community members to be able to join.
  • Able to serve order of 10,000 users. We will not need this scale for the testing, but if the test is successful we want to make sure the system is able to serve large number of volunteers. Also note that this is a very rough estimate based on the different channels and engagements we see. If this number is a blocker, let us know and we can spend some more time further estimating to get a more accurate number.
  • We do not need federation at this stage.

Output
The desired eventual output (partly relying on other tasks linked):

  • the Matrix server (probably Synapse), probably the synapse-admin web UI
  • Element (the standard Matrix web interface)
  • A Matrix-Slack bridge (T382163) - we'd have to figure out what exactly, see task
  • A Matrix-IRC bridge (T382164) - probably matrix-appservice-irc

Event Timeline

leila renamed this task from Deploy a self-hosted Matrix server instance to Deploy a self-hosted public Matrix server instance.Dec 13 2024, 5:10 PM

Is there a domain already selected for hosting this service? Something like matrix.wikimedia.org ?

Not, yet.

(For now we have not started working on the "Test Matrix" task and its subtasks. We have lined them up here so we can have more informed conversations about resourcing this work. If we can prioritize it, you'll see us setting priority for it, assigning it to someone, etc.)

More specifically, what we want is probably

There are two possible approaches: production or WMCS. (There are various SaaS options like EMS or Etke but the amount of bureaucracy involved in using them for a WMF trial makes it not worth it, plus it's very unlikely we'd want to use a hosted solution in the long term, which means it would be nice to gain some experience with self-hosting.)

Using WMCS means extra work if we do end up using Matrix beyond this test, because again it's very unlikely we'd want to use WMCS in the long term. But in the short term it's probably much less work:

  • No need for security review of Synapse etc. since they will be running in an untrusted environment. This assumes we can guarantee on the Slack side that the bridge can only access channels it's supposed to.
  • No need to puppetize things. There is an Ansible playbook (maintained by Etke, one of the more prominent SaaS hosts) that supports just about everything and I imagine using it will cut down the setup effort drastically.
  • It also makes it easier to collaborate with interested non-SRE folks / volunteers.

On the other hand, it might necessitate some sort of legal review (since the bridge will copy some Slack discussions onto the WMCS server - during the test phase these would be dedicated channels only so hopefully not a big deal, but need to think about how to warn people etc). And it would probably need an exception from the WMCS ToU since it technically does not meet the content licensing requirements.

A Mattermost bridge would also be great, to integrate e.g mattermost.wikimedia.de in the long run: https://github.com/mattermost-community/matrix-as-mm

Also avoiding a security review for synapse would be probably better, it is a highly active project with many significant changes in the last few years. WMCS and getting started ASAP to find the challenges in integrating the deployment in different communication channels would be great.

leila updated the task description. (Show Details)

@LSobanski thanks for joining us in the ERC team meeting today. Per our conversation:

  • I assigned the task to you.
  • I updated the task description based on what @Tgr had written above and also some additional conversation we had in the call (some of which after you left). Please edit the task description to fit your needs.
  • We are working on test scenarios per your request. We expect most of the test scenarios focus on the end user testing stage and won't affect the work you all want to do in this task. We will share those scenarios with you anyway (in a week or so) and you can see if there is something there that will affect the hosting.
  • We rely on you and your team to define test scenarios for the components that are operations related.

If you need anything from us, you know where to find us. And thank you!

More specifically, what we want is probably...

I think we'd also need Sygnal so apps can show push notifications.

AIUI some bots/bridges (not sure exactly) don't support end-to-end encryption natively so we might also need Pantalaimon if we want bots or bridging support for E2E channels. This is nice to have at best, E2E wasn't a requirement for the project.

And SUL login with Wikimedia (this is just configuration, see T230536#8105609).

Nice to have: bridges for Discord (matrix-appservice-discord or mautrix-discord?), Telegram (mautrix-telegram) and Mattermost (matrix-appservice-mattermost? or or ask WMDE to bridge on the other side?).

Maybe ma1sd to connect Slack nicknames to their Matrix equivalents? I'm not really sure what exactly it does.