Page MenuHomePhabricator

Explore ideas for workflow support
Open, HighPublic

Assigned To
None
Authored By
Pginer-WMF
Jun 19 2015, 11:18 AM
Referenced Files
F184837: workflow-editor-initial.png
Jun 29 2015, 3:53 PM
F184835: workflow-editor.png
Jun 29 2015, 3:50 PM
F183201: Workflow Pattern Builder(1).png
Jun 24 2015, 4:01 PM
F183199: Workflow Pattern in action.png
Jun 24 2015, 4:01 PM
F183204: Workflow Pattern Builder with AfD.jpg
Jun 24 2015, 4:01 PM
Tokens
"Like" token, awarded by Akuckartz.

Description

Processes defined to coordinate many activities on a wiki rely heavily on conventions and huge instruction texts. More effective processes may facilitate and encourage user participation.

User needs

After initial observations of processes such as Featured article candidates, Wikidata property proposals, Articles for deletion, "Did you know?" nominations, Non-free content reviews, Semi-protected edit requests, Proposed mergers and many others, these processes seem to have some patterns in common (e.g., discussion, consensus, action items, etc.) but they also respond to very different needs. Instead of trying to cover all those specific needs by supporting each process, the idea is to provide tools for the community to easily build support for these processes (and new ones that can appear in the future).

Audience. The audience for workflow support tools are advanced users which are familiar with the policies of the current wiki, but users of different levels of expertise may participate in the different processes.

User research has been conducted (T101331) to learn more about the diversity of the existing processes, and the problems users have supporting them.

Design goals

The basic principles considered for the workflow system:

  • Simple. Make it easy to define and participate in processes. Even if workflows are defined by advanced users, it should not require a PhD on workflow modelling to use the system.
  • Flexible. Processes can vary in complexity (also over time) and the tools to support them should be able to make it possible to support the complex cases without adding an overhead to support the simple ones.
  • Extensible. Since it is not realistic to anticipate al the specific needs, we need to provide extension points where the community can add their own pieces to the puzzle to better support the existing needs or solve new ones.
  • Efficient. Most of the instructions and maul actions can be made part of the system. The goal is not to automate everything, but provide support to leave the decision making to humans and actions to machines as much as possible.

Success metrics. We expect to see the following impact:

  • More participation. By reducing instructions and automating some rules, users will be more likely to participate in processes.
  • More productive. With better support, users should be able to complete processes faster.
  • More processes supported. By making processes easy to define, users will be more likely to provide support for some activities not considered before.

It would be great to instrument some of the existing processes to set a baseline to compare new solutions with.

Design ideas

  • Structured conversations. Workflows can be presented as structured conversations: instead of allowing users to post free text, a predefined structure (i.e., conversation pattern) is expected according to the process. For example, in an process to nominate a page for deletion, it is expected that users will provide the article they are nominating. Present workflows as structured conversations allow to simplify conceptually the transition between regular conversations to workflows.
    • Reusable. These structures can be very specific, but they can also be generic. So it is useful to be able to reuse them for different processes. For example, a process can start with a generic "voting" pattern before a more specific "Nominate feature article" pattern is created.
  • Three main stages: start, replies, and completion. These seem common areas to structure: the initial message that starts the process, the replies that allow the community to participate, and the final resolution of the process.
  • Customisation based on items (data elements, actions and options). Using a form design metaphor, users can define the information captured, the automatic actions to perform (based on the former data) and the options to provide the users. For example, at the start stage we can ask users for an article (data element) and tag it automatically (action); at the "replies" stage a given set of tags to vote "yes" or "no" can be provided (options). The initial items represent a regular flow conversation (title and message to start a conversation, messages to reply, and marking as "resolved" to close the process), and it can be extended with more complex elements to support more advanced workflows as needed.
  • Edit and display states. Each item (e.g., an article input field) has two states. The display state represents the element to communicate what it is (closer to what it will be displayed to the end user). The edit state shows all the information needed to configure that component.
    • Extensible. Although we can provide an initial set of elements, there should be a way for users to extend them: develop new components, and/or have generic scriptable components (e.g., create an action based on a Lua script)
  • Preview to test a workflow. Workflow definition happens at one step of abstraction higher than workflow execution. To make sure that descriptions work as intended, some mechanism for testing them is needed. The testing environment should allow to simulate the use of real data, different users and highlight the invisible effects (e.g., article X is deleted) but do not have real effect on data.

Prototype
You can view a video that illustrate some of the above ideas. The prototype used is also available, but only supports the specific interactions shown in the video.

  • Initially the process structure is the same of a regular flow conversation:

workflow-editor-initial.png (1×1 px, 137 KB)

  • After adding several components a more specific on-wiki process is described:

workflow-editor.png (1×1 px, 166 KB)

Open questions

At the moment the main goal is to validate whether (a) the general approach would be able to capture the needed workflows and (b) it will be easy to use for users that want to create new workflows.
There have been raised other questions worth exploring in the near future:

  • ""Specific items to support.** In the example above you can see that at each stage users can add different kinds of items (page input fields, sets of tags, etc.). Those are based on the initial exploration of workflows, but more thought is required to identify each item and its intended behaviour base on the needs of existing workflows.
  • Properties to support for specific items. In the proposed model, items keep a minimal set of information when edited (e.g., type of item and label/placeholder to determine how to display it), but more advanced properties will be needed for different cases. For example, we may want to provide certain actions only for users with a given role (e.g., to limit who can actually trigger the deletion of a page) or a given period of time. Once those properties are identified they can provided as part of the editing view for each component.
  • Entry points for workflow creation. In the model above the workflow creation process is presented as a way to customise the future conversations of the board. However, a centralised place where all workflows can be useful to keep them in control, especially if it is possible to be shared across processes.
  • Extensibility mechanism. The above model defines several pieces that can be extended (e.g., new data elements, actions and options), but the way to do so has not been defined yet: where and how are these new components created.
  • Sharing conversation patterns across several workflows. A generic pattern such as "voting" can be useful to different workflows. Reusing them make sense, but there are many details to figure out:how to avoid changes on a pattern to affect other workflows using it, how to base one pattern on a previous one without being linked, etc.

Workflow Pattern in action.png (587×706 px, 72 KB)

Workflow Pattern Builder(1).png (2×880 px, 143 KB)

Workflow Pattern Builder with AfD.jpg (2×880 px, 624 KB)

Event Timeline

Pginer-WMF claimed this task.
Pginer-WMF raised the priority of this task from to Needs Triage.
Pginer-WMF updated the task description. (Show Details)
Pginer-WMF added subscribers: Aklapper, Pginer-WMF.
DannyH subscribed.

Neil: we've done something like this before. Templates ... In five years, will people be complaining about the monstrosity?
There's a lot of reliability and flexibility in play. Currently we have a lot of flexibility; this system could add reliability at the expense of flexibility, but editors want and use the flexibility.
Trevor: We probably need something more malleable.
Dan: should I show three versions of it, with different workflows?
Trevor: As you look at 10 or 20 use cases of the 1000, you'll find things that don't fit into this.
Trevor: Templates, at least by passing parameters you can affect the transclusion. Imagine this as a user interface for something that could be done in plain text. Could it be done with JSON or other structured text? (as opposed a specific implementation in the database) As long as you have a system that can interpret that, the builder can be changed easily. Otherwise, the Builder becomes inflexible.
Amir: Brandon presented something like this 3 years ago in Hong Kong. He said that Flow will one day work for workflows, with a domain-specific special programming language.
https://www.mediawiki.org/wiki/Requests_for_comment/UploadWizard:_scale_to_sister_projects
How can this be shared across wikis?
What happens when you edit a workflow that is in use?
Can we use a third-party system, as an example or as a solution?
List of workflow examples. Keywords include rules engine, state machine, workflow. (https://en.wikipedia.org/wiki/Workflow_management_system, https://en.wikipedia.org/wiki/Business_Process_Execution_Language, http://www.infoq.com/articles/bpelbpm)

Neil: we've done something like this before. Templates ... In five years, will people be complaining about the monstrosity?

I've heard many of the complaints about templates, but they have been amazingly successful for the projects. Imagine Wikipedia without infoboxes, citation templates, or hatnotes. There are two main problems:

  1. The original language used was too simple for the required functionality, which resulted in very difficult to maintain templates. Solution: Lua. (See below).
  2. Certain templates do not render well on mobile. Solution: Mostly TBD.

Trevor: Templates, at least by passing parameters you can affect the transclusion. Imagine this as a user interface for something that could be done in plain text. Could it be done with JSON or other structured text? (as opposed a specific implementation in the database) As long as you have a system that can interpret that, the builder can be changed easily. Otherwise, the Builder becomes inflexible.

Yes, I agree the builder should have a backend form that is more flexible. That may also imply not all workflows can be read by the builder (so you may not be able to edit every workflow in the UI). This might be JSON or machine-generated Lua, or some combination.

How can this be shared across wikis?

I don't think we should attempt this until we can share templates across wikis, since this will probably use templates as the output stage (e.g. "This article was nominated for deletion on April 3, 2015. The result was Keep").

What happens when you edit a workflow that is in use?

This is a difficult question we've already started discussing.

Can we use a third-party system, as an example or as a solution?

Maybe. It would have to be extensible to plug into our permissions and actions, so e.g. an admin can write their deletion closing explanation, just click Delete, and it will be done. Does anyone have experience with such a third-party open source flexible rules engine?

Using the design of existing workflow systems to inform our work is also a good idea.

Neil: we've done something like this before. Templates ... In five years, will people be complaining about the monstrosity?

There are extensible systems of many kinds (from templates to gadgets or even browser extensions). I think the problem is not with the idea of extensibility, but with the specific model and its details (e.g., the degree of freedom provided, the tools to create extension, the backwards compatibility, etc.).
I definitely think it is important to note what worked and what did not for templates (Matt mentions limited expressivity and multi-platform support) to identify the aspects to solve, but I don't think that problems in templates imply to discard an extensible model.

There's a lot of reliability and flexibility in play. Currently we have a lot of flexibility; this system could add reliability at the expense of flexibility, but editors want and use the flexibility.

As I described in the "Design ideas" section (and video), I was proposing to provide a simple points that users can extend in multiple ways (with common cases available by default). For example, by default a process is started with a title and a description, but the user can add more pieces to it. Some of these pieces can be provided by us (e.g., an input field to add an article) while others can be created by the community (e.g., a map for users to select a location). Each component is configurable to a different extent depending on the component (e.g., one component can allow you to set the label texts to be displayed, another may be a fully scriptable action). We can control how much to extend is possible by controlling things such as the APIs the kinds of elements to be extended (e.g., UI elements to collect data vs. actions to be performed to them) and tools to do so.

Trevor: We probably need something more malleable.

It would be great to get more specific details. In this broad topic we are exploring many different ideas, so it is really useful to know which ones se to work or not to try with our users.

Dan: should I show three versions of it, with different workflows?
Trevor: As you look at 10 or 20 use cases of the 1000, you'll find things that don't fit into this.

I think the key aspects to verify is whether (a) common cases are easy to support, and (b) edge cases are possible (even if that requires someone to create some kind of extension).

Trevor: Templates, at least by passing parameters you can affect the transclusion. Imagine this as a user interface for something that could be done in plain text. Could it be done with JSON or other structured text? (as opposed a specific implementation in the database) As long as you have a system that can interpret that, the builder can be changed easily. Otherwise, the Builder becomes inflexible.

I think the key aspect is to define "a language" that allows us to describe such processes. That is, the set of concepts that allow them to be described. Those concepts can be represented in visual, textual or both ways, but I think figuring the concepts should go first. Having a text-based format to describe them seems totally reasonable.

Amir: Brandon presented something like this 3 years ago in Hong Kong. He said that Flow will one day work for workflows, with a domain-specific special programming language.

I think the idea makes sense, but I would advise being cautious on the effect that buzzwords tend to have.
We are trying to help users to better go through some in-wiki collaboration activities (most of the time conversational and often related to consensus) that we can classify as part of workflows. However, at the moment we start to talk about "workflows" it seems we need to support any kind of process that can be labelled as "workflow", which seems to be making the problem broader, harder and disconnected to what provides value to our users.
The same happens with "domain-specific languages". As said before, it makes sense to have the process descriptions serialised in a simple form so that it is not coupled with the tools to edit it. It makes sense that it uses the concepts we use to describe such processes (domain-specific), but again, we should not aim to create a programming language (or convey that we are).

What happens when you edit a workflow that is in use?

There are some possible strategies (all have pros and cons), some we discussed:

  • Don't allow process definitions to be changed. Users need to define a completely new process. The tools can make it easy to base it on the existing one to avoid starting from scratch but it is a new process model. New instances will use the new one, old instances will use the old one. It may create fragmentation even for the smaller changes.
  • Apply changes of modified models only to new instances. Changes in a model affect only to new instances. Old instances will follow the former definition. A user going through a list of requests based on different versions will be exposed to different information and behaviour. Mechanisms to avoid confusion may be needed (e.g., warning about instances of old versions).
  • Apply changes to all processes when possible. Not all changes are equal: asking for a piece of information less makes older instances easier to update than adding a new piece of information to ask for a new piece of information more. So it would be possible to discriminate the kinds for changes that can be propagated to the instances and decide one of the above strategies for those that do not.
  • Invalidate old requests. You can always cancel old requests, and ask users to start again with the new process. Users going through that process won't be very happy though.

Can we use a third-party system, as an example or as a solution?
List of workflow examples. Keywords include rules engine, state machine, workflow. (https://en.wikipedia.org/wiki/Workflow_management_system, https://en.wikipedia.org/wiki/Business_Process_Execution_Language, http://www.infoq.com/articles/bpelbpm)

We can definitely learn from existing workflow systems. I had some experience using some workflow engines such as Intalio BPMS Community Edition. They normally support BPMN, and tend to be based on message exchanges across roles. Which support many levels of synchronisation and complex time-based rules that do not seem to apply to the current on-wiki processes but I've not seen supporting discussion and document-related features such as those involved on processes such as "Featured article" reviews.
As I said before, the fact that on-wiki processes can be labelled as "workflows" does not mean we need to support all kinds of workflows.

@Mattflaschen James and Trevor came up with an option in last week's meeting that didn't end up in Joel's notes.

The idea is to use versioning to handle the problems of changing a workflow pattern that's already been used.

I create a new Good Article Review pattern, which is GAR version 1.
People use the pattern to create GAR discussions.
Then I make a change to the GAR pattern, creating version 2.
The existing discussions are still using v1, new discussions will use v2. etc.

I think that would help smooth the transition when there's a change -- you won't suddenly be adding new required entry fields in v2 that existing v1 discussions would have to incorporate.

Could that idea help us solve the edited-workflow problem?

Could that idea help us solve the edited-workflow problem?

Potentially, but if we take this approach we still need to figure out how to list all GAR workflows (from both patterns), and will probably want to allow e.g. filtering on fields that are only in v2.

I was checking with @Capt_Swing, based on the initial research, which design ideas seemed promising, not promising or need to be further evaluated. This is early research but there were interesting outcomes:

  • The idea of conversation patterns (i.e., present workflows as structured conversations) seems to be aligned with the way users perceive processes on wiki.
    • Although the concept seems to work, the term may need some initial clarifications.
    • Considering a regular conversation the base case that "evolves" into a workflow as more structure is added seems to work as a way to connect the concepts of regular conversations and wiki-workflows.
    • Although reusing these patterns makes sense, it seems that the focus is on creating very specific solutions for the needs each community has. Thus, the later should be probably presented in a more prominent way compare to the reuse facilities.
  • The three main stages defined (start, replies and completion) seem to be enough to capture the overall structure of the structured conversations, although support for more complexity is needed in each of this general structure.
  • The types of elements for each section (data elements, actions and configuration options) seem generic enough to support the needs discovered so far but we need to test it with a more realistic process (example).
    • Notifications seem to be a key concept so they may have a specific weight (either as a specific kind of action for the users to use or as a prominent part of the configuration of every other component).
    • The idea of adding components instead of presenting all initially seems affective since different components need very different options.
    • Making it informative at a glance is important. We need to be careful with which information is shown when configuring vs. displaying a component added by the user. Information density may not be a big concern if users get relevant information that provides them a clear idea of the process they are defining.
  • There is not much information on the need for a dedicated way to preview workflows based on the current research. The intention is to facilitate a more iterative process (encouraging users to explore and try new components without fear of breaking anything) and allowing to test early versions of processes without involving more people (preview allows to switch to different users). However, a good point was raised by Jonathan, that maybe a simple option to create a copy of the process under the user namespace can be enough to cover the user needs. So more research on how there processes are tested currently and which issues they find in that process (if any) will be helpful.
  • Considering a regular conversation the base case that "evolves" into a workflow as more structure is added seems to work as a way to connect the concepts of regular conversations and wiki-workflows.

If we allow changing the pattern for an existing page, we need to bear this in mind when working on various features. E.g. some features may only display workflow items (e.g. 'featured picture nomination' being one workflow item) that match the current pattern.

In T103082#1500350, @Mattflaschen wrote:
  • Considering a regular conversation the base case that "evolves" into a workflow as more structure is added seems to work as a way to connect the concepts of regular conversations and wiki-workflows.

If we allow changing the pattern for an existing page, we need to bear this in mind when working on various features. E.g. some features may only display workflow items (e.g. 'featured picture nomination' being one workflow item) that match the current pattern.

I think there are two separate aspects to consider here:

  • Relationship between "regular" flow discussions and "structured" workflows. We could present these concepts as completely different concepts or present them as related concepts. Even if internally they are supported very differently, the idea of presenting them as conversations plus structure (thus, "workflows" being an evolution of "regular" flow conversations) seems to align with how users perceive them.For example, iIf we imagine a simple workflow based on discussion plus support/oppose voting, I think users would benefit from the discussion to work in the same way that other "regular" flow discussions work, and not in a completely different way just because it is part of a workflow.
  • Flexibility in the transition between concepts. Even if we consider workflows to be "conversations plus structure", it does not imply that the structure could be added at any point in time or freely added/removed anytime. Personally I think that it would be positive to provide some way to iterate in the definition of a process (as opposed to hope for processes to be perfectly defined on day one and never evolving). Finding a versioning model that works well in this context (i.e., with asynchronous instances with long livespan involving multiple users) is not an easy task. We could start simple and consider that modifying a process definition results in an new version for the new instances created and the old ones are "archived" and behave according to the old definition until they are completed. When we present that to users that are familiar with workflows on wiki, we may get a better idea if the approach seems promising or we are missing something.
  • Relationship between "regular" flow discussions and "structured" workflows. We could present these concepts as completely different concepts or present them as related concepts. Even if internally they are supported very differently, the idea of presenting them as conversations plus structure (thus, "workflows" being an evolution of "regular" flow conversations) seems to align with how users perceive them.For example, iIf we imagine a simple workflow based on discussion plus support/oppose voting, I think users would benefit from the discussion to work in the same way that other "regular" flow discussions work, and not in a completely different way just because it is part of a workflow.

Yes. Another alternative is for regular discussions to just be a particular, built-in, structured workflow pattern. I think this is how it was originally envisioned (doesn't mean we have to do it like that, though).

I believe there will be little community support for the proposal described here. It's not useful. We're not looking for structured discussions, and we don't want to go to a Flow board to paste in an article title for deletion. Twinkle lets us nominate an article for deletion while at the article page.

A much better starting point is something like this sketch:
https://upload.wikimedia.org/wikipedia/mediawiki/a/a7/1-_Workflow_Pattern_Builder.jpg
that could replace community-scripts such as Twinkle.

I believe there will be little community support for the proposal described here. It's not useful. We're not looking for structured discussions, and we don't want to go to a Flow board to paste in an article title for deletion. Twinkle lets us nominate an article for deletion while at the article page.

Simply stating that "we" want, or don't want, something is overstating your case... YOU might not want this. I, on the other hand, do.

The current system of workflows (on en.wp) to do various activities (discuss article deletion, and nominate for 'did you know' being among the most common) have been built the way they are by using the technical functions allowed by the software, not because it is a "good" way to do it.

The fact that you recommend using a specialist community-developed tool as a workaround for the current official procedure says to me that the current official procedure is too difficult for most people to use.

Wittylama wrote: Simply stating that "we" want, or don't want, something is overstating your case

I clearly began by saying I believe... about what the Community would want. The purpose of posting on Phab is to alert the devs that discussions are needed before building anything. I believe we are all unanimous that the worst case scenario for everyone is when time and money get invested in writing code, and that code winds up in the trashbin. Dev's work is too valuable. I've been talking on Wiki with staff about WMF-Community engagement to sort out how WMF&Community want to proceed. I could be mistaken, but I suspect Community Consensus will be that the Prototype-approach here is inferior to Twinkle. I'd like to see a Workflow solution that is superior to Twinkle.

The Design goals are good. The Design ideas and Prototype need discussion.

I had a lightweight proposal for how to store workflows, documented here: https://www.mediawiki.org/wiki/Requests_for_comment/Workflow

This format is pretty easy to edit, and I think gives us a good balance of clarity, flexibility, and extensibility. It's not a Turing-complete language or anything, it allows signals, states, and actions to be wired together but the actions are strictly defined by access controls and can only be created in code, through a code-review process.