If it's going to remain installed, we should implement Herald support at least.

In T127640#2129282, @hashar wrote:

Looks like the best argument to not bother disabling Conpherence. It does no harm and some likes to use it.

I think this is a pretty terrible argument.

Let's say I did not even try to argue but instead I have took a decision. Might not please an handful of people who don't even use Conpherence...

• Re T127640#2116945: "less intimidating than IRC" has become moot as we use Zulip for GSoC and Outreachy participants.
• Re "messaging problematic users in Phab itself": It's not a common action nowadays. The workaround is using the on-wiki email function (which not every user has enabled, I admit).
• Re Discussing Differential changes in T127640#2116327, in the meantime it has been decided to not use Differential in T191182.
• Re "used for a lot of private messages" in T127640#2116130, the usage numbers don't support that:
  • Number of users who have performed any actions in Conpherence in February 2019: 20
  • Number of users who have performed any actions in Conpherence in March 2019: 27
  • Total number of Conpherence channels which have seen any activity in February 2019: 16
  • Total number of Conpherence channels which have seen any activity in March 2019: 18
• Re T127640#2129282 , I consider fragmentation a harm. An example are well-intended but rather empty rooms like https://phabricator.wikimedia.org/Z632 with a very vague scope, where people expect to receive help but do not receive it. Or https://phabricator.wikimedia.org/Z716 and https://phabricator.wikimedia.org/Z604 where folks want to receive help or report a bug instead of creating a task. Or people who post random stuff in random unrelated channels like in https://phabricator.wikimedia.org/Z707.
• https://phabricator.wikimedia.org/Z308 (Phab updates / Phab interested parties) has been superseded by https://www.mediawiki.org/wiki/Talk:Phabricator/Help and https://phabricator.wikimedia.org/phame/blog/view/14/

Looking at the list of active channels, the only both valid and active use case currently seems to be https://phabricator.wikimedia.org/Z398 (discussing and syncing site requests) where another venue would need to be found (though https://phabricator.wikimedia.org/T222018#5142792 sounds like sometimes IRC is used?). I am wondering if these discussions could take place in the tasks themselves?

Given the bullet points above I lean towards disabling Conpherence. 'Disabling' does not exist though:

• Uninstalling an application creates a "404 Not Found" page when trying to access a Conpherence chat.
• Setting "Can Use Application" to "Administrators" creates a "You Shall Not Pass: Restricted Application" page when trying to access a Conpherence chat.
• In any case, data would not get deleted in the database but data would become inaccessible via a web browser.

I'm not sure yet what's the best way out / how to provide a heads-up when it comes to allowing Conpherence users to 'archive' their data.

In T127640#5154388, @Aklapper wrote:

Looking at the list of active channels, the only both valid and active use case currently seems to be https://phabricator.wikimedia.org/Z398 (discussing and syncing site requests) where another venue would need to be found (though https://phabricator.wikimedia.org/T222018#5142792 sounds like sometimes IRC is used?). I am wondering if these discussions could take place in the tasks themselves?

As the process owner for SWATs (where most of those site requests are handled) I would also prefer to have relevant logistics on the tasks themselves (within reason, of course) so as to give the SWAT deployer all the info they might need.

Given the bullet points above I lean towards disabling Conpherence. 'Disabling' does not exist though:

• Uninstalling an application creates a "404 Not Found" page when trying to access a Conpherence chat.
• Setting "Can Use Application" to "Administrators" creates a "You Shall Not Pass: Restricted Application" page when trying to access a Conpherence chat.
• In any case, data would not get deleted in the database but data would become inaccessible via a web browser.

I'm not sure yet what's the best way out / how to provide a heads-up when it comes to allowing Conpherence users to 'archive' their data.

I would:

• Announce we're going to disable Conpherence with a 6 week warning.
• At six weeks, set to only admins can use. This is just in case someone needs something that's only in Conpherence.
  • Do not advertise this step :) People need to understand the 6 week deadline to be hard.
• 4 weeks later, uninstall.

+1 generally, thanks Andre.

At six weeks, set to only admins can use. This is just in case someone needs something that's only in Conpherence.

For the records (as this might create wrong expectations): Phab administrators can not bypass object access policies.
So when a Conpherence chat room is access-restricted, admins won't be able to access it either (now or in the future) via the UI. Data is still in the database though.

Re "messaging problematic users in Phab itself":

This is what notifications are for. Mentioning a user by @name or subscribing them will notify them. If users disable or don't read notifications that's a social problem you would have with any notification system.

Re Discussing Differential changes

Indeed we decided to stop using Differential.

Announce we're going to disable Conpherence with a 6 week warning.

Sounds like a good plan, +1

In T127640#5162133, Dzahn wrote:

Re "messaging problematic users in Phab itself":

This is what notifications are for. Mentioning a user by @name or subscribing them will notify them.

Indeed. For the records, https://www.mediawiki.org/wiki/Phabricator/Help/Managing_mail covers that
"You can also distinguish when you were explicitly mentioned in a comment by filtering X-Phabricator-Mail-Stamps on mention(@yourusername)."

For the records, I sent a heads-up announcement to wikitech-l@ at https://lists.wikimedia.org/pipermail/wikitech-l/2019-May/092063.html and also posted in the most active room at https://phabricator.wikimedia.org/Z398 - if you have more places in mind where this should be posted, please share them here!

Phabricator comes with a wide array of tools, trying to cover everything that you would expect from a forge / communication hub / project management software. Given the amount of resources their developer community has, by necessity these tools tend to be mediocre at best, and Conpherence is no exception (visually unappealing, poor notification support, poor search). On the other hand, they are well integrated with each other (same authentication system / user identity / ACL, same UX, same markup, same API, can use things like {T123} which self-update as needed). A reasonable argument can be made both for adopting them fully, and for using best-of-breed tools for everything (Maniphest probably meets that standard but other parts of Phabricator don't) and spend effort on integration.

De facto I think we have gone for the latter by sticking with Gerrit and not including Harbormaster to the CI shortlist, and to some extent with making Discourse an official project (although we seem to be sticking with the Phame blogs). So I think this decision fits into our general approach (and using a loose set of good tools instead of a tightly integrated set of mediocre ones would be my personal preference as well).

Security team has an ongoing chat with the stewards via conpherence. So far we (Security) have proposed a move to IRC but there are various blockers. I wonder if in the close-up-shop case (I guess restrict with no uninstall in this case) for general use there could be an exception for acl*stewards and acl*security-team. Stewards in particular have a variety of technical capability and phab and esp conpherence has been really valuable in meeting in the middle while keeping back and forth easy and secure.

@greg @Aklapper

Could we restrict creating new rooms to just Trusted-Contributors etc, There are valid and useful rooms, Site-Requests seems to be heavily used and per chase security is using it.

I don't see why we are shutting down a whole system when it it doesn't appear to be majorly abused or negatively used at the moment.

In T127640#5178820, @Peachey88 wrote:

Could we restrict creating new rooms to just Trusted-Contributors etc, There are valid and useful rooms, Site-Requests seems to be heavily used and per chase security is using it.

That should probably be the case if we decide to keep Conpherence enabled, to be honest. It should be the basic level of protection for the service.

I don't see why we are shutting down a whole system when it it doesn't appear to be majorly abused or negatively used at the moment.

See: T127640#5154388 :)

In T127640#5176939, @chasemp wrote:

Security team has an ongoing chat with the stewards via conpherence. So far we (Security) have proposed a move to IRC but there are various blockers. I wonder if in the close-up-shop case (I guess restrict with no uninstall in this case) for general use there could be an exception for acl*stewards and acl*security-team. Stewards in particular have a variety of technical capability and phab and esp conpherence has been really valuable in meeting in the middle while keeping back and forth easy and secure.

@greg @Aklapper

Interesting, I wasn't aware (see also: one of the negatives of Conpherence, even admins can't access rooms they don't have explicit access to :) ). I think this is worth discussing more what can be done here.

So far we (Security) have proposed a move to IRC but there are various blockers.

I think this is worth discussing more what can be done here.

Maybe we could start by listing these blockers a bit more specifically.

In T127640#5154388, @Aklapper wrote:

• Re T127640#2116945: "less intimidating than IRC" has become moot as we use Zulip for GSoC and Outreachy participants.

@Aklapper should we look at making zulip more widely available to the broader technical community? It seems like a really good chat platform from my initial evaluations.

In T127640#5181169, @mmodell wrote:

should we look at making zulip more widely available to the broader technical community? It seems like a really good chat platform from my initial evaluations.

Similar opinion here... CC'ing @srishakatux who's in my understanding the main contact for Zulip.

In T127640#5176939, @chasemp wrote:

Security team has an ongoing chat with the stewards via conpherence. So far we (Security) have proposed a move to IRC but there are various blockers. I wonder if in the close-up-shop case (I guess restrict with no uninstall in this case) for general use there could be an exception for acl*stewards and acl*security-team. Stewards in particular have a variety of technical capability and phab and esp conpherence has been really valuable in meeting in the middle while keeping back and forth easy and secure.

@greg @Aklapper

Interesting, I wasn't aware (see also: one of the negatives of Conpherence, even admins can't access rooms they don't have explicit access to :) ). I think this is worth discussing more what can be done here.

Comments from a steward in that chat regarding this (I'll let them name themselves here if they want):

I don't think IRC will be a good alt. to phab conpherence! at least here if you send
message then all users (online/offline) can see it now, tmw..etc, but on IRC if you are
not online then you will not see it (unless you ask for record)

In T127640#5178889, @Dzahn wrote:

Maybe we could start by listing these blockers a bit more specifically.

The general gist is above in the comment from a steward. IRC's limitations of no history, no alerting, etc. Conpherence requires only a web browser (the same tooling as the wiki) for those things. Conpherence does well if requirements are within this narrow scope. It's also easy to keep to a limited subset of folks.

It's possible we are at a stage where a mailing list makes sense, the conpherence thread has become useful enough to want to keep the dialogue going but none of us in security team feel pedantic about it. Until we have time to discuss that with the parties involved and can agree the loss of real time chat is fine it makes sense to keep conpherence for at least these two groups as the cost of keeping it seems very low.

In T127640#5178820, @Peachey88 wrote:

There are valid and useful rooms

@Peachey88: Please provide references for your statement, apart from Site-Requests (and what chasemp brought up). Because I am not aware of any examples...

Site-Requests seems to be heavily used and per chase security is using it.

@Peachey88: Please explain the term "heavily used". I see zero messages in April, two off-topic messages in March, and one "Hello' message by a now-blocked user account in February in the Site-Requests chat room.

In T127640#5181344, @chasemp wrote:

The general gist is above in the comment from a steward. IRC's limitations of no history, no alerting, etc. Conpherence requires only a web browser (the same tooling as the wiki) for those things. Conpherence does well if requirements are within this narrow scope. It's also easy to keep to a limited subset of folks.

May be of interest: T222458: Evaluate Element as recommended IRC client
It can handle permanent presence, alerting, it has a web browser, desktop, iOS and Android version.

I make heavy use of IRCCloud (for more than 5 years) but I would prefer something other than IRC due to it being inferior with netsplit and etc.

Basically I'd like to keep access to stuff via web browser, no need to worry about the netsplits and unexpected disconnections, direct complete log access (no 3rd party for browsing log is no-no) at a minimum.

In T127640#5183628, @revi wrote:

I make heavy use of IRCCloud (for more than 5 years) but I would prefer something other than IRC due to it being inferior with netsplit and etc.

Basically I'd like to keep access to stuff via web browser, no need to worry about the netsplits and unexpected disconnections, direct complete log access (no 3rd party for browsing log is no-no) at a minimum.

I think that is a laudable goal. Something related is Mozilla currently evaluating alternatives/replacements to their use of IRC: http://exple.tive.org/blarg/2019/04/26/synchronous-text/ & http://exple.tive.org/blarg/2019/05/03/goals-and-constraints/ & http://exple.tive.org/blarg/2019/05/14/the-next-part-of-the-process/

However, it's mostly tangential to this conversation. If we wanted to do a similar process to Mozilla's I think we should, but we shouldn't leave Conpherence enabled "just in case" we might do that evaulutation in some speculative future. :)

In T127640#5176939, @chasemp wrote:

Security team has an ongoing chat with the stewards via conpherence. So far we (Security) have proposed a move to IRC but there are various blockers.

@chasemp: Would it be possible to define and lay out communication requirements for this case, preferably in public? (Though might be worth a separate task?)

@Aklapper I've started a discussion on stewards-l, it might take a bit of time. (Maybe worth a new task, indeed.)

Thank you!

No consensus at stewards-l, one suggesting Mailing list and other one (in conphrence) suggested use of somewhat chat-like system contradicts, so the best conclusion I can draw is no consensus for anything right now.

@revi (and @chasemp): Stewards may be the only group currently 'really' using Conpherence and having one solution per group does not make much sense, I guess... :)
In the discussion about potential other tools, which options have been proposed and discussed? For example, has Zulip been investigated? Could the communication requirements which Stewards have be shared somehow/somewhere?

This is quotes from my email on stewards-l

• No loss of log while my device is sleeping.
• No dependency on 3rd party tools to browse logs.
• No requirement for a new program to use it.

Point 1 and 2 is inter-connected as if you don't need to browse 3rd party logs to browse logs you probably don't lose logs when you disconnect at one point and then reconnect later.

Zulip could be the alternative but haven't used it yet.

@chasemp @revi @Aklapper - just FYI, we're still actively discussing this within the stewards/Security-Team conpherence chat. Could someone on the Security-Team get administrative access to Zulip so that we could set up a secure test chat there to evaluate as an alternative? Or can I file a bug for someone with Zulip admin access to do that? Right now, Zulip seems like it might be the most promising alternative for the stewards/Security-Team use case.

@sbassett: Do you already have an account in the Zulip instance used by Wikimedia?

@Aklapper - just registered one w/ my wikimedia.org email address.

In T127640#5260781, @Aklapper wrote:

In the discussion about potential other tools, which options have been proposed and discussed? For example, has Zulip been investigated? Could the communication requirements which Stewards have be shared somehow/somewhere?

If you are interested in longer-term options, do check out T186061: Evaluate Matrix / Element as the recommended chat system for Wikimedia. A trial will launch in a few days, we could involve the stewards in that. (Note though that the trial itself will not be suitable for sensitive private discussions, it's just a tech demo.)

In T127640#5181169, @mmodell wrote:

@Aklapper should we look at making zulip more widely available to the broader technical community? It seems like a really good chat platform from my initial evaluations.

So, far, we have used Zulip only for outreach programs (mostly GSoC & Outreachy) to stay connected with students and interns before, during, and after the program. It has worked great for us, and we have ~1000 users who are currently signed up on the platform. Before pushing the same instance for broader use, I would hesitate and would like to figure out:

• How many users we would be able to support on the current instance?
• Migration from Zulip's cloud service to our servers
• Pros and cons of having one instance over multiple instances for all Wikimedia use cases
• Promoting the platform differently

In T127640#5263309, @sbassett wrote:

@Aklapper - just registered one w/ my wikimedia.org email address.

@sbassett I see that you've just joined Zulip, welcome :) With our current organization settings, you or any member or admin on Zulip will be able to create a private stream and add or invite members to it. Even, though I've shared some concerns above, I think it is okay for your team to test Zulip and have team specific discussions there for now. Your insights might also help learn if or when we should think about promoting Zulip (the same instance if at all) more broadly.

Ok, thanks, @srishakatux!

Would it be possible for Security and stewards to take a look and evaluate Zulip in the next two weeks, to get an idea if that could work for you?
(And if you think that Zulip won't work, explain what's missing?) Thanks!

Can I also be added to zulip for testing? (Email address (my phab handle)@(my phab handle).wiki)

Nevermind, seems like I can self-register.

@revi - I think you can just sign up with an email, google or github account here: https://wikimedia.zulipchat.com/register/. Once you have an account, we should set up a stewards/secteam channel and try to add everyone else.

@Aklapper - I think that's a very reasonable amount of time for us to try out Zulip and make a decision - thanks!

If it needs a bit more than to weeks that's also fine; I think we just need an arbitrary deadline so this task does not become neverending. Thanks a lot everybody for your understanding! :)

@Aklapper - we now have a solution in place (an existing irc channel) as a replacement for the secteam-stewards conpherence chat. We may re-evaluate Zulip or Matrix/Riot as a solution further down the road, but for now, the existing irc channel should be fine. Out of curiosity - with the disabling of the conpherence feature, will its database content also be flushed? It might be nice to store an archive of the secteam-stewards conpherence chat somewhere if this is the case.

In T127640#5293428, @sbassett wrote:

we now have a solution in place (an existing irc channel) as a replacement for the secteam-stewards conpherence chat. We may re-evaluate Zulip or Matrix/Riot as a solution further down the road, but for now, the existing irc channel should be fine.

Very happy to hear that! Thanks a lot to everybody who was involved in finding a replacement (and willing to make some compromises, I guess)!

Out of curiosity - with the disabling of the conpherence feature, will its database content also be flushed?

See the last bullet points in comment T127640#5154388 - please ask if they are not clear. :)

It might be nice to store an archive of the secteam-stewards conpherence chat somewhere if this is the case.

@Aklapper re:

In any case, data would not get deleted in the database but data would become inaccessible via a web browser.

This sounds fine, but is there any planned deadline for this data to persist? Or just "indefinitely"? Thanks.

@sbassett: Indefinitely, if I understand your question correctly: The data in the database itself will not get deleted. After switching off Conpherence, the data could only be accessed anymore by someone having permissions to run SQL queries (and after constructing that SQL query).

It might be nice to store an archive of the secteam-stewards conpherence chat somewhere if this is the case.

People who have access to that Conpherence chat room could save the content of that Conpherence chat room from within their web browser, I guess, however I don't know how acceptable that is when it comes to legal aspects of such non-public conversations.

Indefinitely, if I understand your question correctly: The data in the database itself will not get deleted. After switching off Conpherence, the data could only be accessed anymore by someone having permissions to run SQL queries (and after constructing that SQL query).

Yep, thanks. That works.

People who have access to that Conpherence chat room could save the content of that Conpherence chat room from within their web browser, I guess, however I don't know how acceptable that is when it comes to legal aspects of such non-public conversations.

We've done this, for now, as a backup measure of our own. The textual data is being stored in a safe, protected space.

As announced in https://lists.wikimedia.org/pipermail/wikitech-l/2019-May/092063.html and described above, this has now happened:

Conpherence has become unavailable for most users (as I changed the "Can Use Application" setting for Conpherence from "Public" to "Administrators").

Big thanks everyone who helped by commenting or by moving to a different system to coordinate work! (I need an inverted version of https://xkcd.com/927/ now.)

If you missed this and need something that is only in Conpherence, Phab Administrators might help you to get that something for another four weeks (August 7th, 2019). I'm using the word "might" because Phab administrators are not as almighty as you may think.

• announced https://lists.wikimedia.org/pipermail/wikitech-l/2019-July/092283.html
• updated https://www.mediawiki.org/w/index.php?title=Phabricator%2FHelp&type=revision&diff=3308801&oldid=3305419
• updated https://www.mediawiki.org/w/index.php?title=Phabricator%2FUsing_Conpherence&type=revision&diff=3308803&oldid=2835239

Keeping this task open until we've completely disabled Conpherence in four weeks.

Change 542787 had a related patch set uploaded (by Aklapper; owner: Aklapper):
[operations/puppet@production] Phabricator: Uninstall Conpherence application also in default settings

https://gerrit.wikimedia.org/r/542787

Change 542787 merged by Jcrespo:

[operations/puppet@production] Phabricator: Uninstall Conpherence application also in default settings

https://gerrit.wikimedia.org/r/542787