Page MenuHomePhabricator

Request creation of "chat" VPS project (for a Mattermost instance)
Closed, ResolvedPublic

Description

We would like to request an install of Mattermost for volunteers to support a Slack-like chat capability, particularly in the context of the COVID-19 pandemic and the increased importance of online conferences and editathons. Among other things, chat functionality can be used to support "office hours" during an online editing campaign extending over a week or a month, and can also be useful as an alternative to video for some conference sessions as well as for general planning meetings between volunteer groups.

This category of meeting-related tools could informally be described as a social suite.

See also Wikimedia Meet, a Jitsi install: T249159

Event Timeline

Sounds like roughly the same goal that the matrix project has (T193964). Maybe y'all could collaborate there rather than starting a new project? This is an area that @Tgr has long been interested in.

Sounds like roughly the same goal that the matrix project has (T193964). Maybe y'all could collaborate there rather than starting a new project? This is an area that @Tgr has long been interested in.

Yes, got in touch with @Tgr first, he is no longer interested in the matrix project for the moment, because installs of matrix hosted elsewhere are apparently sufficient for his needs. I do feel that Mattermost on VPS would offer us some additional benefits for broader volunteer programs, though.

As always with projects that will attract free-form outside users, we have several concerns/caveats:

  • Please be aware that maintenance and support of these new services will not be in any way the responsibility of the cloud-services team or WMF staff. If things break and your users complain to us we will send them to you.
  • Please be sure that any users of this service are greeted with a banner or welcome message that links to the code of conduct, and also identifies this as a volunteer-run project without official WMF support, and specifying that there is no guarantee of privacy for any communication that takes place there.
  • Refrain from logging access or traffic (including the conversations themselves)
  • Finally, and most importantly: no one from the WMF will be enforcing our code of conduct, or monitoring to ensure respectful communication. For that reason, admins of this project should be on notice that any indication of CoC violation will result in immediate project shutdown. https://www.mediawiki.org/wiki/Code_of_Conduct

If those terms are agreeable (and you have some idea of how to implement the banner/welcome message), then this request is approved. Will you also be needing a public IP address for access?

Aklapper renamed this task from Request creation of "chat" VPS project to Request creation of "chat" VPS project (for a Mattermost instance).Jun 17 2020, 4:39 PM

Oh, one other question: why 'chat' and not 'mattermost' if it's going to host mattermost?

Why not Matrix ?

This question is off topic for this ticket @Akuckartz. You are basically walking up on a stranger and asking why they are wearing a jacket and not a sweater. The question may be interesting to you, but it is not material to the creation of a Cloud VPS project.

Oh, one other question: why 'chat' and not 'mattermost' if it's going to host mattermost?

Mostly because the product (chat rooms) should not be coupled to the underlying technology, basically we might change from mattermost to some new fancy thing that might come in the future.

@bd808 I think that your comparison is neither helpful nor appropriate - especially considering previous comments mentioning Matrix (including one by yourself) and the openness of others to consider alternatives.

Yes, got in touch with @Tgr first, he is no longer interested in the matrix project for the moment, because installs of matrix hosted elsewhere are apparently sufficient for his needs. I do feel that Mattermost on VPS would offer us some additional benefits for broader volunteer programs, though.

It might be helpful to know which "additional benefits" are expected by "Mattermost on VPS".

@Akuckartz: This task is not about discussing dozens of potential real-time chat systems to be hosted somewhere. This task is specifically about creating a VPS project for a Mattermost instance. Hence @bd808 is correct when pointing out that the scope of the task shall not be broadened.

See T184606: Evaluate and set up a test instance of FOSS persistent chat software as a companion to Q&A system for communication with third-party developers instead for high-level conversations about real-time chat systems - thanks!

As always with projects that will attract free-form outside users, we have several concerns/caveats:

  • Please be aware that maintenance and support of these new services will not be in any way the responsibility of the cloud-services team or WMF staff. If things break and your users complain to us we will send them to you.

I know and I accept the responsibility for now. Hope others come and help :)

  • Please be sure that any users of this service are greeted with a banner or welcome message that links to the code of conduct, and also identifies this as a volunteer-run project without official WMF support, and specifying that there is no guarantee of privacy for any communication that takes place there.

Probably someone needs to write a mattermost bot: https://docs.mattermost.com/developer/bot-accounts.html It should not be too hard. Does this solution work for you?

  • Refrain from logging access or traffic (including the conversations themselves)

As far as I checked (and I might be wrong). The conversations are kept in the database for search purposes (mattermost allows you to search in previous discussions which comes really handy sometimes, WMDE uses mattermost for internal discussions). We might need to write a maintenance cron job to remove old discussions to comply with data retention policy (any discussion older than a month for example) but I don't know if this is good enough for you.

  • Finally, and most importantly: no one from the WMF will be enforcing our code of conduct, or monitoring to ensure respectful communication. For that reason, admins of this project should be on notice that any indication of CoC violation will result in immediate project shutdown. https://www.mediawiki.org/wiki/Code_of_Conduct

This is a little complicated. I have two points on different angles:

  • A violation of CoC should not be a reason to shut down a project, there are violations of conduct policies in every project. Of course, trying to enforce it is doable (albeit it never can fully cover every case). The CoC you are referring to has a committee to enforce it (yours truly a member) so would giving rights to the committee to receive the complaints and enforce it a good enough compromise?
  • The CoC you mentioned is only for technical spaces. A chat in IRC channel of #wikipedia-fa on freenode is not covered by TCoC by chats in #mediawiki channel is. We can apply the same rule here and cover technical channels under the CoC or make an exception and cover all of it by that. If we go with the former, the problem of enforcing a conduct policy on non technical spaces arises and if we go with the latter, the committee needs to agree on it (and possibly an amendment on the CoC itself as outlined in the policy. The problem with the latter is non-technical channels will bring people from outside the technical community and while the committee knows how to handle e.g. a case of doxxing in general (we got trainings and such) but they don't know the history of people and previous cases in other places (because they are sent to T&S or the wiki admins instead of us) which limits their ability to respond properly (basically a non-technical user gets a clean paper to write to). It's not a huge problem (lots of non-technical people also make comment in phabricator too) but it's a drawback

If those terms are agreeable (and you have some idea of how to implement the banner/welcome message), then this request is approved. Will you also be needing a public IP address for access?

Checking the installation guide I don't think we need an IP address but it needs a database to connect to (to keep the discussions, users, etc.) and while I can put the database on the VM, I don't want to have a nightmare every time we move to a new VM. A while back I requested Trove to be installed and used in the cloud (T212595: [Feature request] Database as a Service (Trove) for Cloud VPS projects) but I haven't seen much progress on it. An interim solution would be to mount it to somewhere persistent (I think it's possible, isn't it?)

As always with projects that will attract free-form outside users, we have several concerns/caveats:

  • Please be aware that maintenance and support of these new services will not be in any way the responsibility of the cloud-services team or WMF staff. If things break and your users complain to us we will send them to you.

I know and I accept the responsibility for now. Hope others come and help :)

ok!

  • Please be sure that any users of this service are greeted with a banner or welcome message that links to the code of conduct, and also identifies this as a volunteer-run project without official WMF support, and specifying that there is no guarantee of privacy for any communication that takes place there.

Probably someone needs to write a mattermost bot: https://docs.mattermost.com/developer/bot-accounts.html It should not be too hard. Does this solution work for you?

Yep, a welcome bot would be fine. It will need to be installed early in the project's life.

  • Refrain from logging access or traffic (including the conversations themselves)

As far as I checked (and I might be wrong). The conversations are kept in the database for search purposes (mattermost allows you to search in previous discussions which comes really handy sometimes, WMDE uses mattermost for internal discussions). We might need to write a maintenance cron job to remove old discussions to comply with data retention policy (any discussion older than a month for example) but I don't know if this is good enough for you.

Purging conversations according to the data retention policy (iow, deleting anything older than 90 days) should be adequate.

  • Finally, and most importantly: no one from the WMF will be enforcing our code of conduct, or monitoring to ensure respectful communication. For that reason, admins of this project should be on notice that any indication of CoC violation will result in immediate project shutdown. https://www.mediawiki.org/wiki/Code_of_Conduct

This is a little complicated. I have two points on different angles:

  • A violation of CoC should not be a reason to shut down a project, there are violations of conduct policies in every project. Of course, trying to enforce it is doable (albeit it never can fully cover every case). The CoC you are referring to has a committee to enforce it (yours truly a member) so would giving rights to the committee to receive the complaints and enforce it a good enough compromise?

I don't think there should be any expectation that any WMF staff time will be spent in support of this project. If you're able to muster enough volunteer time to keep things civil (or that simply isn't an issue) then there won't be any problems.

I didn't mean to imply that I would shut down the project for a single violation; rather that if it becomes a CoC-free haven we'll have no choice but to shut it down, as managing in-project discourse is well outside of the capabilities of the WMCS team.

  • The CoC you mentioned is only for technical spaces.

All communication that takes place on WMCS infrastructure is goverened by the WMCS terms of use and the technical spaces Code of Conduct.

  • The CoC you mentioned is only for technical spaces.

All communication that takes place on WMCS infrastructure is goverened by the WMCS terms of use and the technical spaces Code of Conduct.

Then I can assume the issue of CoC is resolved. We give the rights to the committee and they are responsible for enforcing it. Does that make sense?

aborrero moved this task from Inbox to Approved on the Cloud-VPS (Project-requests) board.
aborrero added subscribers: Andrew, aborrero.

This has been approved. I will create the project soon.

aborrero triaged this task as Medium priority.Jul 1 2020, 3:42 PM

Thanks. What about the database request? :D

Thanks. What about the database request? :D

We do not have persistent volumes (yet). The only solution for this today is an NFS share. And you really do not want to be running a db off of an NFS share.

Thanks. What about the database request? :D

And you really do not want to be running a db off of an NFS share.

It'll be sooo fun (not for the users I assume).

Jokes aside: Thanks. I will try to have a persistent VM and one that I throw away more often.

Mentioned in SAL (#wikimedia-cloud) [2020-07-02T09:57:09Z] <arturo> create project with ladsgroup (Amir1) as projectadmin (T255046)

Yes, got in touch with @Tgr first, he is no longer interested in the matrix project for the moment, because installs of matrix hosted elsewhere are apparently sufficient for his needs. I do feel that Mattermost on VPS would offer us some additional benefits for broader volunteer programs, though.

To clarify, given the federated nature of Matrix (which means that all Matrix servers are entry points into the same chat network), I don't see any privacy or security advantages in running our own instance on Wikimedia Cloud. Using a well-chosen third-party instance (I haven't investigated but chat.privacytools.io looks promising, for example) would very likely serve Wikimedia community members better in terms of security, privacy, performance or availibility. Even if some Wikimedian group is interested in setting up their own Matrix homeserver (the Hungarian technical community had some discussions about that), it's not clear if Wikimedia Cloud is the best place for that, given how hostile its terms of use is to running anything user-facing, even if it's not really enforced in practice.

(For the same reason, and for reasons of mission and value alignment, I personally don't think it makes a lot of sense to set up Mattermost or another walled-garden chat tool on Wikimedia Cloud, either; but I don't feel strongly enough about that to try to dissuade others from trying. This task would not be the appriopriate venue for that, in any case.)

@bd808 Hey, it's now accessible in https://chat.wmcloud.org like early days of gmail, it's invite-only right now but anyone who joins can invite anyone so it should cover lots of people.

So:

root@chat-mattermost01:/var/log/exim4# tail mainlog 
2020-08-15 20:16:23 Start queue run: pid=30514
2020-08-15 20:16:23 End queue run: pid=30514
2020-08-15 20:19:22 1k72eA-0007wT-GH <= <> H=(chat.wmcloud.org) [::1]:49940 I=[::1]:25 P=esmtp S=5098
2020-08-15 20:19:22 1k72eA-0007wT-GH => ladsgroup@gmail.com R=smart_route T=remote_smtp S=5229 H=mx-out01.wmflabs.org [172.16.1.239] I=[172.16.1.207] C="250 OK id=1k72eA-0001q6-J1" DT=0s
2020-08-15 20:19:22 1k72eA-0007wT-GH Completed
2020-08-15 20:26:23 Start queue run: pid=30628
2020-08-15 20:26:23 End queue run: pid=30628
2020-08-15 20:34:43 1k72t1-0008Av-5Z <= <> H=(chat.wmcloud.org) [::1]:49960 I=[::1]:25 P=esmtp S=5098
2020-08-15 20:34:43 1k72t1-0008Av-5Z => ladsgroup@gmail.com R=smart_route T=remote_smtp S=5229 H=mx-out01.wmflabs.org [172.16.1.239] I=[172.16.1.207] C="250 OK id=1k72t1-00028h-8G" DT=0s
2020-08-15 20:34:43 1k72t1-0008Av-5Z Completed

(But I never receive the emails)

@bd808 Hey

Please file new Phabricator tasks to report bugs rather than taking messages on to existing tasks.

I'm working on a welcome message, can you provide me something?

Not without more context, no.

It seems mails from this node is not reaching outside

It looks like the emails may be malformed? Google is rejecting delivery. (Ignore the ipv6 connectivity error in the log output. That's not the real problem.)

$ ssh root@out01.cloudinfra.eqiad.wmflabs
$ cd /var/log/exim4
$ zgrep 1k72eA-0001q6-J1 mainlog.2.gz
2020-08-15 20:19:22 1k72eA-0001q6-J1 <= <> H=chat-mattermost01.chat.eqiad1.wikimedia.cloud (chat-mattermost01.chat.eqiad.wmflabs) [172.16.1.207]:59870 I=[172.16.1.239]:25 P=esmtp S=5351
2020-08-15 20:19:22 1k72eA-0001q6-J1 H=gmail-smtp-in.l.google.com [2607:f8b0:400d:c03::1b] Network is unreachable
2020-08-15 20:19:23 1k72eA-0001q6-J1 ** ladsgroup@gmail.com R=dnslookup_unsigned T=remote_smtp_unsigned H=gmail-smtp-in.l.google.com [209.85.201.26] I=[172.16.1.239] X=TLS1.2:ECDHE_ECDSA_CHACHA20_POLY1305:256 CV=yes DN="C=US,ST=California,L=Mountain View,O=Google LLC,CN=mx.google.com": SMTP error from remote mail server after end of data: 550-5.7.1 [185.15.56.18      14] Messages missing a valid address in From:\n550 5.7.1 header, or having no From: header, are not accepted. q15si7590040qvf.69 - gsmtp
2020-08-15 20:19:23 1k72eA-0001q6-J1 ladsgroup@gmail.com: error ignored
2020-08-15 20:19:23 1k72eA-0001q6-J1 Completed

@bd808 Hey

Please file new Phabricator tasks to report bugs rather than taking messages on to existing tasks.

Sure

I'm working on a welcome message, can you provide me something?

Not without more context, no.

II was referring to this from this ticket:

  • Please be sure that any users of this service are greeted with a banner or welcome message that links to the code of conduct, and also identifies this as a volunteer-run project without official WMF support, and specifying that there is no guarantee of privacy for any communication that takes place there.

If those terms are agreeable (and you have some idea of how to implement the banner/welcome message), then this request is approved. Will you also be needing a public IP address for access?

I will follow up on the email issue somewhere else.

I'm working on a welcome message, can you provide me something?

Not without more context, no.

II was referring to this from this ticket:

  • Please be sure that any users of this service are greeted with a banner or welcome message that links to the code of conduct, and also identifies this as a volunteer-run project without official WMF support, and specifying that there is no guarantee of privacy for any communication that takes place there.

https://wikitech.wikimedia.org/wiki/Wikitech:Cloud_Services_Terms_of_use#If_my_tools_collect_Private_Information... says you should say at least:

By using this project, you agree that any private information you give to this project may be made publicly available and not be treated as confidential.

By using this project, you agree that the volunteer administrators of this project will have access to any data you submit. This can include your IP address, your username/password combination for accounts created in Cloud Services services, and any other information that you send. The volunteer administrators of this project are bound by the Wikimedia Cloud Services Terms of Use, and are not allowed to share this information or use it in any non-approved way.

Since access to this information is fundamental to the operation of Wikimedia Cloud Services, these terms regarding use of your data expressly override the Wikimedia Foundation's Privacy Policy as it relates to the use and access of your personal information.

Per prior discussion you should also reference https://www.mediawiki.org/wiki/Code_of_Conduct as applying to all messages, both public and private, sent using the service.