Page MenuHomePhabricator

Implement Phabricator Spaces
Closed, ResolvedPublic

Description

Phabricator Spaces are policy containers for groups of objects or, in plain English, areas where just some users with the appropriate permissions can access and work. Think for instance about spaces for Security, Procurement, Fundraising, Legal... accessed only by the members of those teams and optionally special guests. The full specifications are described in https://secure.phabricator.com/T3820#116875

The Phabricator maintainers expect to have a first version of Spaces ready by the end of June 2015, probably to be fine tuned during July after the first users report feedback.

Once this feature is implemented, the request of a new space should be a simple process, and could be possibly be handled by the Project-Admins process to request a new project. AffCom, Community Liaisons, and Fundraising Tech are the first cases that come to mind, and it is clear that these teams handle some private information combined with their public activities.

The implementation of Spaces might imply the deprecation of the Security extension that we are running locally in Wikimedia Phabricator, which currently allows the handling of security bugs and special requests to Operations. To be discussed.

The #Engineering-Community team is driving this initiative in Wikimedia's side. We are pooling a budget in the Community Engagement department to fund this development. See the WMF Short Form Business Case for Phabricator Spaces.

Old description

NOTE: Marking this task as Stalled. We need some months of real use of Phabricator, including private tasks, before even defining a plan for easy to use private projects that can scale. For now Phabricator is only for public projects that eventually have private tasks (for security reasons, etc). Until further notice, full private projects can stay in Trello, Asana, Google Drive, your mailbox, etc.

There are teams dealing with sensitive information that need to work with private tasks. Their theoretical options are:

Short term

Request a special security policy for their tasks, which will imply a new entry in the Security dropdown menu. Just like the Security team now or the Operations team as soon as RT is migrated, they will need to work privately in a fully open context. They will be responsible of maintaining the list of project members with access to the private tasks, and also who else can access to the tasks via CC. The Wikimedia Phabricator maintainers will not be responsible of any accidental leaks due to human errors messing with policies or users.

Note that we don't have any process to address a request like this. If a team steps in, we will need to discuss the proposal from scratch.

Mid term

Wait until upstream implements Spaces. Then you will be able to request an own namespace with default private policies fitting your needs. You will manage who can access to these namespaces and you will be able to create your projects within that namespace.

Related Objects

StatusSubtypeAssignedTask
ResolvedQgil
ResolvedQgil
ResolvedQgil
Resolved RobLa-WMF
ResolvedQgil
ResolvedDzahn
ResolvedQgil
ResolvedAklapper
Invalid mmodell
Resolved mmodell
Resolved mmodell
DeclinedQgil
Declinedgreg
ResolvedRobH
DeclinedVarnent
ResolvedAklapper
Resolved mmodell
Resolved mmodell
ResolvedAklapper
ResolvedQgil
ResolvedRobH
ResolvedAklapper
ResolvedAklapper
ResolvedAklapper
ResolvedAklapper
ResolvedAklapper
DeclinedAklapper
ResolvedQgil
Resolved chasemp
Resolved chasemp
Resolved chasemp
Resolved chasemp
Resolved chasemp
Resolved chasemp
ResolvedQgil
Resolved gpaumier
ResolvedAklapper
ResolvedDzahn
ResolvedDzahn
DeclinedNone
InvalidRobH
DuplicateRobH
Declined mmodell
Duplicate mmodell
ResolvedQgil
Resolved mmodell
Resolved Springle
ResolvedNone
Resolved mmodell

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

There is an ongoing discussion about the possibility for the Fundraising-Tech team to get permissions to mark some tasks private. See T88762: Enable select Fundraising people to modify policy for tasks.

Seeing the discussion at T88762 and also knowing that other teams are considering Phabricator if the problem of private tasks can be solved (Zero, some non-tech WMF teams, and even the Affiliations Committee)... I wonder whether we should consider to simply add more options to the "Security" field.

The solution taken for security bugs and Ops private tasks is basically what these teams need. I don't see any problems in this approach, other than a longer dropdown menu and probably an increased risk of misfiled tasks.

In T823#1030131, @Qgil wrote:

Seeing the discussion at T88762 and also knowing that other teams are considering Phabricator if the problem of private tasks can be solved (Zero, some non-tech WMF teams, and even the Affiliations Committee)... I wonder whether we should consider to simply add more options to the "Security" field.

The solution taken for security bugs and Ops private tasks is basically what these teams need. I don't see any problems in this approach, other than a longer dropdown menu and probably an increased risk of misfiled tasks.

Not opposed to adding more items to the drop down. I think but for the most part it is hard to make it flexible and not overwhelm people with weird and implicit behavior. We will be at four items pretty soon and from there it will start to get muddy if we aren't careful. I think before we go that route we should try a few case by case and see how things are used.

A few thoughts are:

  • Drop down items in the security menu can do lots of things, and are not necessarily obvious to the choosers we have found.
  • We can actually create template tasks that will carry policy over with them to any tickets created using the template. I investigated this with upstream a bit to see if the auxiliary fields would populate with a template (they don't), but most everything else does. So if we set up a template that sets view/edit policy as fundraising ...anyone in that group (I believe) can use it to create tasks that are secure. Only a few people in that arena need to be hip to how to navigate policy with rights to un-hide things as appropriate, and fix edge cases.
  • I think there are not many use cases we haven't encountered that we are aware of? Maybe HR and other non-technical people will have them, but I'm wondering if this is the right place for that kind of stuff. Not trying to keep them out but this is a bit of the wild west for HR type personal data.
  • I think a few Fundraising folks with policy knowledge, a template task for easy creation, and circling back in a few weeks to see how this is working is a reasonable way to feel our way through the darkness here.
In T823#1030131, @Qgil wrote:

Seeing the discussion at T88762 and also knowing that other teams are considering Phabricator if the problem of private tasks can be solved (Zero, some non-tech WMF teams, and even the Affiliations Committee)... I wonder whether we should consider to simply add more options to the "Security" field.

The solution taken for security bugs and Ops private tasks is basically what these teams need. I don't see any problems in this approach, other than a longer dropdown menu and probably an increased risk of misfiled tasks.

More detail on the AffCom use case-- they are discussing tools to easily manage their internal approval process for user group applications. User group applications should only be visible to AffCom members. Phabricator looks like it has great task management tools for them if it's possible for them to make their project private.

  • Drop down items in the security menu can do lots of things, and are not necessarily obvious to the choosers we have found.

Absolutely true, and a good reason to find alternatives to the drop-down menu, especially for cases where most if not all the private tasks will be created by a single team (as opposed to users creating them, as it is the case for security bugs and ops requests).

  • We can actually create template tasks that will carry policy over with them to any tickets created using the template.

This is great news indeed.

  • I think there are not many use cases we haven't encountered that we are aware of?

Not many, but I have stopped some myself, pointing the teams who asked to this task. Now that there is progress, I will encourage them to declare their needs in tasks, and add them as blocked by this one.

  • I think a few Fundraising folks with policy knowledge, a template task for easy creation, and circling back in a few weeks to see how this is working is a reasonable way to feel our way through the darkness here.

Strong +1. Fundraising-tech can go first; the rest can wait and see.

Research & Data also uses private trello boards for handling sensitive requests (for example, private data requests that may involve an NDA, grant proposals etc). It sounds like the mid term solution would be the best for us.

@Qgil: I understand the mid-term solution is becoming long-term, how do we move forward with the short term approach?

@DarTar, I would say that as soon as @atgo confirms that she is happy with T88762: Enable select Fundraising people to modify policy for tasks we could create the same type of setup for other teams.

@Qgil, great – can you let me know when that happens and open a similar request for R&D?

@DarTar @Qgil 'm pretty happy with it, though I would like to expand the
group who can write policy a little bit to include a few more people. I'll
follow up with Chase about that when I have a chance.

According to this estimate, asking Phacility Ink to prioritize the development of private spaces in Phabricator would cost about $18K USD. How I see it, for (say) $30K and a bit of extra work from our side, we could stop maintaining our own Security hack, make happy Fundraising Tech, Zero, Operations, and probably other Engineering & Product teams, and bring to Phabricator the non-tech WMF teams using Asana, Trello, or more rudimentary ways to organize their sensitive work.

@greg, please consider budgeting for this contract work in the next fiscal year. I think it would be a good investment and I will support it in any forum where someone is interested in my opinion. ;)

"global, default-deny walls around groups of unprivileged users"?

Hmm. Not sure if that is entirely what we want.

@Krenair Its been renamed to "Implement top-level "Spaces" that provide policy isolation to groups of objects" and that, I believe, is what we want.

The upstream implementation is focused on the use case of a design studio who uses phabricator and want to have a separate phab instance for each client. Essentially it's would allow employees to switch between namespaces but clients would be jailed to just one namespace. So most of the phab database would be unique to each namespace with just a few things shared among all of them. It would be perhaps overkill for our use case and I don't know if it would satisfy our very particular rules about who can and cannot see a security bug's task.

Then again I might be wrong, perhaps it would work somehow ... I don't see how it would work for security bugs but it might work well for the other use cases.

I think that use case works well for our teams dealing with sensitive information. According to https://secure.phabricator.com/T3820#86217, there will be a main space (public, like this instance now), and then specific spaces with specific policies. Projects and tasks can be moved from one space to the other. A task created publicly could be moved to a private space and vice versa.

Our local Security extension has allowed us to migrate Bugzilla and RT without waiting for a proper solution from upstream (that as of today still does not exist). However, since the beginning we agreed that this would be an interim solution, being the final solution either an upstream implementation (or our implementation being evolved and upstreamed, theoretically). Spaces is how Phabricator upstream plans to solve this problem, if they would have developed them a year ago we would have used them. I think we should support its development if we can afford it.

Qgil changed the task status from Stalled to Open.May 12 2015, 7:03 AM
Qgil claimed this task.
Qgil raised the priority of this task from Lowest to Low.

I will try to write a proposal and pool the budget necessary for Phacility Inc (the Phabricator maintainers) to work on Spaces. Dob't expect too much progress in the next couple of weeks (before the Wikimedia Hackathon), though.

In T823#1278602, @Qgil wrote:

I will try to write a proposal and pool the budget necessary for Phacility Inc (the Phabricator maintainers) to work on Spaces. Dob't expect too much progress in the next couple of weeks (before the Wikimedia Hackathon), though.

Is there a specific place we can give feedback on what tasks we'd like to pay upstream to work on? I think a lot more people would benefit if we asked upstream to fix search rather than private projects which would affect very few people.

Qgil raised the priority of this task from Low to High.May 28 2015, 1:30 PM

Is there a specific place we can give feedback on what tasks we'd like to pay upstream to work on?

Currently not, and it is a good idea. It would make sense to have a relation between the relevance of a task and the likeliness of us paying to have it fixed, therefore I would start by discussing priorities of tasks under "Wikimedia requests" at the Phabricator (Upstream) workboard.

For instance, I'm increasing the priority of this task, since I want to have the budget part solved before the end of June / end of of WMF fiscal year.

While private projects might affect less people than search, the impact on these teams would be very high, and I think it would ultimately benefit our support on Phabricator. For instance, if the percentage of WMF teams using Phabricator as primary tool increases, and they also happen to find problems with search, then the chances of raising the priority of the related tasks and even finding budget to pay for the fixes would also increase.

Qgil renamed this task from Process to request a private project to Implement Phabricator Spaces.Jun 3 2015, 9:09 AM
Qgil updated the task description. (Show Details)
Qgil edited projects, added Phabricator (Upstream); removed Phabricator.

This exists since the phabricator upgrade yesterday:

https://phabricator.wikimedia.org/spaces/

The database for it had to be created that's why i know it wasn't there before.

There is also a button "Create Space" now, it's limited to admins though.

https://phabricator.wikimedia.org/spaces/create/

From https://secure.phabricator.com/T8376#118034

Spaces is now a real (prototype) application, but doesn't do anything useful yet.

Contract Request Form sent to WMF-Legal. This will take some days but will not stop the development upstream.

After the last Wikimedia Phabricator update, https://phabricator.wikimedia.org/spaces/ exists. According to upstream it is "useless" as of today. Development continues upstream at a fast pace.

On our side, I got the contract past the Legal check and now it is waiting for the Financial check. Should be fine, since we have the budget.

In T823#1355913, @Qgil wrote:

After the last Wikimedia Phabricator update, https://phabricator.wikimedia.org/spaces/ exists. According to upstream it is "useless" as of today.

I got Phab-03 up with Phabricator at upstream's HEAD if it is any use to anyone else. Spaces seems to be working quite well there. I'll just be testing on it for a few weeks (at least) and then'll probably delete it unless someone needs it.

Qgil lowered the priority of this task from High to Medium.Jun 12 2015, 10:26 AM

(Lowering priority to Normal only to reflect that others are taking the initiative. My part as pusher is basically done. I keep reacting fast to anything I can do with the contract and the project upstream.)

The contract has been approved by Legal and Finance. After the trivial step of collecting a couple of signatures, Phacility Inc can invoice us by the end of this month. Therefore, I will stop talking about bureaucracy here. Back to 100% technical discussion. :)

Krenair closed subtask Restricted Task as Resolved.Jun 19 2015, 2:46 PM

From https://secure.phabricator.com/T3820#123094 :

Spaces are now available in master. They'll promote to stable in this week's release, and are likely to leave prototype next week.

You can find some documentation here: https://secure.phabricator.com/book/phabricator/article/spaces/

T8376 has details and additional context, as well as links to remaining work.

This means that we might get stable Spaces functionality in our next upgrade, or in the following at most (meaning in about 2-4 weeks).

We can start thinking who will want to test Spaces...

Phab-03 was ready for Spaces testing (I say was because T103918 seems to have left it in a peculiar state).

Alright, we have Spaces. I'm resolving this task, and I'm happy to give a hand to the first team willing to test them.

I'm wondering if the people using Spaces know what they're doing... I've been getting subscribed to tasks in spaces that I can't actually view, and it's actually kind of annoying (and concerning that someone might be trying to bring my attention to something but me not being able to view).

I've been getting subscribed to tasks in spaces that I can't actually view

Meh. :(
Please point people to the documentation which has always been stating:

A Space's policy is absolute and stronger than any other policy rules. If a user cannot see a space, the user can never see objects inside the space either, even if they are author, assignee or subscriber of the task in that space. (To allow users which are not member of the space to view or edit an object in the Space, a Custom Policy needs to be applied on the object instead of a Space.)

Or if you have an idea how to make that documentation more clear, feel free to edit.

I've been getting subscribed to tasks in spaces that I can't actually view

Please point people to

Part of the problem is that I can't identify the people doing it. We can't even send out a mass message to all users of private spaces warning them to clean up their act, because in at least one case all you can see about a space visibility policy is that it's set to custom - unless your account fits the space policy (and maybe the space edit policy?) in which case you can probably view the policy itself.

Part of the problem is that I can't identify the people doing it.

@Krenair: As you wrote "I've been getting subscribed to tasks in spaces that I can't actually view", how did you find out? We currently have two restricted spaces in use (CEP and Ops) so teams could be contacted (both have a mailing list) to raise awareness.

The notification counter goes up without any new unread notifications being shown. It's an upstream bug, which they could fix... But still wouldn't deal with my concern that maybe some of these are people actually trying to add me to things (or maybe not... who knows?).

The notification counter goes up without any new unread notifications being shown.

Happened half a dozen times to me!

Counting notifications that you aren't able to see is problematic... other than that I'm not sure what could be done about @Krenair's larger concern.