Page MenuHomePhabricator

GSoD 2020: Week 6, 7, 8, & 9
Closed, ResolvedPublic

Description

This task is a break-down of the sixth and seventh week of GSoD based on the timeline T255360: GSoD 2020 Proposal: Improving Wikimedia's onboarding processes and documentation standards

Related tasks for other weeks can be found here T262918: GSoD 2020

Week 6 - 7 (October 19 - 30)
  • Revise the proposed format based on feedbacks gotten from mentors and documentarians.
  • Create a document guide explaining the finalized format discussed in depth.
  • Discuss how the format documented can be enforced on documentarians and across all Wikimedia documentations.

Last week, I proposed a documentation review process and explained how tools like phabricator can be leveraged on for documentation reviews.
This week, I'll be focusing more on revising the suggested process based on feedbacks gotten, I will be making adjustments where necessary. But firstly, I would like to restructure the five review processes into something less verbose.

Revised review process

The documentation review process has been revised into 3 basic processes after which documentations would only be approved after all processes have been undergone.

1) Prototype/Self review

The writer of a documentation is always the first reviewer of their own documentations, therefore, a documentarian is obligated to review and do a copy editing of his documentation before submitting it for review.
As a part of this step, the documentarian is allowed to share the documentation with a friend or colleagues.

The documentarian is to make sure that the documentation follows Wikimedia's documentation style guides, formatting, methodologies, and use of language. This helps the reviewer identify his own mistakes and makes sure the documentation conforms with the style guide. Additionally, new documentation tasks are to be created by strictly following this guide.

2) Technical Review

This phase would require a senior colleague to go through the documentation and check for technical accuracy, completeness of its content, and how easy it would be for new contributors to adapt. Comprehensive responses and feedbacks would be gotten here.

3) Non-technical Review

Here, the documentation has been checked for technical accuracy and correctness and it's time to ascertain that the style guides, formatting, methodologies, and use of language were strictly followed. This would be done by an experienced reviewer who is more familiar with these guides.

Documentations would only be moved out of the review phase if all and only if all review stages has been passed.

Phabricator as a documentation review tool

Phabricators workboard can be very resourceful for reviewing documentations, here is a detailed summary of how it can be used.

Workboards are divided into columns, with each column containing related tasks.

  • A new column for documentations that are currently reviewed will be created on the workboard. This column will be named in-review.
  • A new task will be created for all documentation that needs a review, the task would describe what the documentation is about and provide a link to it.
  • All newly created documentation review tasks will have a tag named needs-review, and will be added to the in-review column on the documentation workboard.
  • A tag will be used to identify the review stage a documentation is currently in. There will be 3 tags namely;
    • prototype review
    • technical review
    • non-technical review.
  • A documentation that has passed all review stages will be moved to the Completed column on the documentation workboard.

The need for a documentation review admin.

The described structure and review workflow above further creates more questions like;

  • How do people find documentation that needs a review?
  • Who decides who is to review a documentation?
  • Who decides when a documentation is fit to move to the next review stage, and who moves it?
  • How do documentarians find the right reviewers for their newly created documentation?
  • etc

The idea of a review admin is to have someone who will oversee the entire review life cycle of a documentation.
The review admin will be responsible for;

  • Making sure a newly created documentation has a review task created for it on phabricator
  • The newly created task has the needs-review tag which indicates that the documentation needs a reviewer
  • The task is moved to the in-review column on the documentation workboard.
  • Contact Engineers and documentarians who would do the review of the documentation and also oversee and partake in this review process.
  • Occasionally do the non-technical review phase of a documentation.
  • Determine when a documentation is fit to be moved from one review stage to another.

Identifying documentations in review

Documentations are not created on phabricator, therefore there has to be a way to identify documentations that are being reviewed currently, that have been reviewed, or that needs a review.

For this purpose, templates can come in really handy. There is a ( {{ Draft }} ) template for documentations that are currently in development, I think this should be maintained.

Then we will need to create new templates, one for in-review and another for reviewed.
The meaning of both templates is immediately obvious, the first will be used for documentations that have passed the development phase and are currently being reviewed, while the other will be used to identify documentations that have been reviewed and published.

Documentation review life-cycle

The below is a flow chart showing the review life-cycle of a documentation.

Documentation review life cycle.png (1×2 px, 317 KB)

Feature requests

Phabricator and Mediawiki aren't documentation review tools, therefore, using them to review a documentation would most certainly come with some setback. One of such setback is fostering effective communication between the reviewer and the documentarian. While it's true that phabricator and discuss pages can be used for effectively communicating, they might not be the best option for reviewing a documentation.

Proposing Review Issues

When a reviewer goes through a document and finds a phrase, sentence, or line he would like to change, seek more clarification on, or ask a question about. He should not have to go to phabricator to type out the phrase just so he could reference it. Rather, he should be able to highlight that phrase or sentence and create a discussion thread for it. This discussion thread is called a review issue.

This can pose a lot of advantages to how documentations are being reviewed, all the discussion to any specific sentence being reviewed will reside in one single place and are automatically referenced.

Issues should have the following features;

  • When reviewing a document, all highlighted text that has an associated issue should have their background colored. When you click on any of them, the discussion thread of that issue should be displayed.
  • Issues should be sharable, which means there should be a share button that generates a sharable link to that issue.
  • Issues should be resolvable. After a series of discussions on any issue thread, the reviewer should be able to mark the issue has resolved.
  • Reviewer should be able to highlight a sentence or phrase and suggest a better sentence or phrase.
  • The documentarian should be able to view the reviewers suggestions and approve it or start a discussion thread on it.
Todo
  • Include a section that helps people understand what types of documention this review process would be a great fit for.
  • Prototype/Technical reviews should be done by the writer, teams, or group who wrote the document
  • review-admin should do only as much work a necessary.

Event Timeline

@Gbahdeyboh ~ thanks for working on this. I have a few thoughts and suggestions--nothing you have to integrate but may want to think about.

  1. I'd suggest writing something that helps people identify what kinds of documentation projects the process will work best for. New documentation and major rewrites or editions may benefit from it. While smaller changes and updates can probably go forward outside of it.
  1. Prototype/Technical review -- I would consider thinking about these two reviews as being the responsibility of the writer and the team or working group working on the technology being documented. They should identify who is best for the technical review, as they will likely have the best understanding of the technology.

If there is a documentation admin, I think the expectation should be that they help manage the process rather than the technical reviewers. I do think it is fine for the admin to help recruit reviewers for the non-technical review (grammar, style, etc) because this does not require specific knowledge of technology and will conceivably be easier for a wider group of folks to work on (including folks who are less technical).

Overall, I would try to reduce the amount of work a separate "admin-review" person would be responsible for and consider making the person who has written the documentation responsible for moving it through the full process (chaning tags, creating subtasks, finding reviewers, etc). We don't know who will take on this role, and in my experience, it could be easy for the review process to fall through unless you have a real dedicated stakeholder in this role.

  1. Along with tasks, I would consider thinking about creating subtasks for the different review stages. So the main task (the documentation) would have a subtask for each of the protype/technical/nontechnical reviews -- so these can be resolved individually rather than switching out tags at each stage of the process (which can be easy to forget). @Aklapper may be a good person to consult at this stage to ask about the best way to refine this part of the process.

I think that is it for now-- I may add some additional thoughts here later.

Thanks a lot for the feedbacks @srodlund.

  1. It's very true that these review process might not be needed for all kind of documentations, I would include a section stating what documentation these processes might be good to adopt as a review process.
  1. I also think it is important that the review-admin should do only as much work a necessary, since there will be multiple documentations being reviewed, it makes sense to minimize the amount of work the review-admin does.
  1. I totally agree, creating subtasks for each review step means each step can be individually resolved. However, the Idea of switching tags was suggested because I felt it would be nice to be able to know the review status of a task by just looking at the task tag on the documentation workboard without actually opening it first. @Aklapper, I would really appreciate your inputs and thoughts regarding this.

I would work on all changes suggested and revise this document.

My comments might be a bit lost, as I'm not enough into this topic and as I have no overview what already happened in the weeks before. :)

On a very high-level:
If I started in Phabricator and look at Documentation at https://phabricator.wikimedia.org/project/profile/987/ , I guess I'm supposed to find out that I should click the second link (to https://www.mediawiki.org/wiki/Documentation) as the first and third don't really help me? That page lists a bunch of links under "pages" though it's not clear to me who is / if I am the target audience. The "guidelines" link to a page called "Technical documentation prioritization" which seems to be mostly about creating tasks in Phabricator but not how I could get involved as a technical writer? What's the funneling concept when landing? Where's the place that I am supposed to look at as a "new" technical writer in the Wikimedia world and stuff I'm supposed to know and follow? It's probably useful to have a review concept but I must be able to find it and people must know that they are supposed to use and follow it.

Proposing Review Issues

As it (rightfully) says that people "should not have to go to Phabricator", I don't see yet how that could be implemented or where that would live. The general current scheme to provide feedback about a [documentation] page is to bring it up on the Discussion page of that very [documentation] page. Which in my very personal opinion does not scale as no individual will watch/follow all and any discussion pages to get notified about new "review issues". Feedback venue consolidation has been an unsolved problem for many years in the Wikimedia movement. In my ideal world, many discussion pages would be redirects to single, centralized feedback venues that people could watch/follow.

  • Phabricators workboard can be very resourceful for reviewing documentations, here is a detailed summary of how it can be used.
  • Workboards are divided into columns, with each column containing related tasks.
  • A new column for documentations that are currently reviewed will be created on the workboard. This column will be named in-review.

The current Documentation Phab workboard seems to use "Categories" as columns, described at https://www.mediawiki.org/wiki/Phabricator/Project_management#Typical_examples_for_uses_of_workboards . Workboard uses should not and cannot be mixed: there should not be columns to both track progress (e.g. "in-review") and track categories on one single board as a task can only be in one column on a workboard.

A new task will be created for all documentation that needs a review, the task would describe what the documentation is about and provide a link to it.

I'm not sure I understand this, probably as it lacks a subject who is acting on this. Maybe this means "Create a dedicated task for a dedicated piece of documentation work"? If so then I agree, and this is currently covered under the strangely named page https://www.mediawiki.org/wiki/Documentation/Technical_documentation_prioritization#Filing_a_technical_documentation_request

All newly created documentation review tasks will have a tag named needs-review, and will be added to the in-review column on the documentation workboard.
A tag will be used to identify the review stage a documentation is currently in. There will be 3 tags namely;

  • prototype review
  • technical review
  • non-technical review.

I don't understand how "in-review / needs-review" and "prototype review / technical review / non-technical review" relate to each other.

  • A documentation that has passed all review stages will be moved to the Completed column on the documentation workboard.

Swimlanes / progress should preferably be expressed via workboard columns (dragging cards from left to right as progress is made).
Subtasks might be another implementation option but that is way heavier - if I understand correctly (not sure, see my previous question) it sounds like someone would have to create a subsubtask (needs review) of a subtask (in review) of a task (the doc item work itself) and I really don't see anyone wanting to spend time on that and deal with that notification noise level and that fragmentation of conversations in three different tasks about the very same topic (work on a doc item).

Progress should never be expressed via Phabricator project tags. In an ideal world, project tags are supposed to be rather static from the beginning to the end of a task (except for when correcting mistakes, like assumptions which codebase project needs to be touched to fix the task, or tagging teams). Tags should not express status. (We unfortunately make that mistake in a few places like Community-consensus-needed or Patch-For-Review, but we shall not repeat this mistake).

Hi @Aklapper Thanks a lot for the thorough feedbacks. It's very much appreciated.

Which in my very personal opinion does not scale as no individual will watch/follow all and any discussion pages to get notified about new "review issues".

Yes! You're right! No one will definitely watch all discussion pages to get notified about review issues. However, review issues are meant to create a sleek communication between the reviewer and the documentarian. Therefore, only the reviewer and documentarian will get notified per document. Only the documentarian gets notified when a new issue is created on that specific document he authored, and a reviewer will only get a notification when an issue created gets a response. That way, both reviewer and documentarian only get notifications only when their attention is really needed.

Workboard uses should not and cannot be mixed: there should not be columns to both track progress (e.g. "in-review") and track categories on one single board as a task can only be in one column on a workboard.

Yes! They aren't mixed, it's a single review task per documentation. The category the task belongs to on the workboard is what changes. There are only two categories in which a documentation review task can be in.

  1. "in-review" category - For documentations currently being reviewed or one which needs a review
  2. Completed - This is for all tasks that has been completed.

Maybe this means "Create a dedicated task for a dedicated piece of documentation work"?

Yes!

I don't understand how "in-review / needs-review" and "prototype review / technical review / non-technical review" relate to each other.

"in-review / needs-review" are tags used to indicate if a documentation needs a review or if is currently being reviewed.

"prototype review / technical review / non-technical review" are the review stages a particular documentation needs to go through.

The tags only let you know which documentation is yet to be reviewed and which is undergoing a review.

I wasn't aware that tags are meant to be static, using workboard columns to express progress is a really great Idea!

if I understand correctly (not sure, see my previous question) it sounds like someone would have to create a subsubtask (needs review) of a subtask (in review) of a task (the doc item work itself) and I really don't see anyone wanting to spend time on that and deal with that notification noise level and that fragmentation of conversations in three different tasks about the very same topic (work on a doc item).

The sub-tasks were actually suggested for each review stage (technical review / non-technical review) rather than each review status (in-review/needs-review). But they won't be necessary anymore as @srodlund suggested that the prototype and technical review should be done by the team, individual or group working on the document(which is a really great idea). This means the reviewer is left with just one review stage (non-technical review) which Ideally would only need to occupy a single column on the documentation workboard.

However, review issues are meant to create a sleek communication between the reviewer and the documentarian. Therefore, only the reviewer and documentarian will get notified per document.

It's currently unclear to me how that would be implemented and where.
In any case: Relying on single individuals creates bottlenecks and basically never scales.

Only the documentarian gets notified when a new issue is created on that specific document he authored,

It's unclear to me where and how "Review Issues" will be created. In general, anyone interested should be able to get notifications on whatever they want. And that's how MediaWiki itself already works. (PS: General note: Please feel encouraged to replace "he" with "they" in sentences, for some more inclusiveness. :)

and a reviewer will only get a notification when an issue created gets a response.

That would exist in the MediaWiki software by watching a talk page, and in Phabricator as tasks.

Workboard uses should not and cannot be mixed: there should not be columns to both track progress (e.g. "in-review") and track categories on one single board as a task can only be in one column on a workboard.

Yes! They aren't mixed, it's a single review task per documentation. The category the task belongs to on the workboard is what changes. There are only two categories in which a documentation review task can be in.

  1. "in-review" category - For documentations currently being reviewed or one which needs a review
  2. Completed - This is for all tasks that has been completed.

I either don't understand, and/or we might be getting lost in terms here. One Phab project has one workboard. A workboard has columns. Columns can be used either for progress tracking or for categorizing tasks. I don't know what you mean by "the category the task belongs". Categories of tasks usually do not change. See https://phabricator.wikimedia.org/tag/documentation/ : The column headers are current categories.

I don't understand how "in-review / needs-review" and "prototype review / technical review / non-technical review" relate to each other.

"in-review / needs-review" are tags used to indicate if a documentation needs a review or if is currently being reviewed.

"prototype review / technical review / non-technical review" are the review stages a particular documentation needs to go through.

My question was how they are related. Both things sound like stages; I don't see any difference.
My understanding would rather be "needs-review 🡒 in prototype review 🡒 in technical review 🡒in non-technical review"; if at all.

The sub-tasks were actually suggested for each review stage (technical review / non-technical review) rather than each review status (in-review/needs-review).

I don't see a difference between stage and status. A status is a stage and a stage is a status to me. :)

But they won't be necessary anymore as @srodlund suggested that the prototype and technical review should be done by the team, individual or group working on the document(which is a really great idea). This means the reviewer is left with just one review stage (non-technical review) which Ideally would only need to occupy a single column on the documentation workboard.

A project workboard should never use workboard columns as a mix of categories (current use of columns on Documentation) and progress ("status"; "stage").

It's currently unclear to me how that would be implemented and where.
In any case: Relying on single individuals creates bottlenecks and basically never scales

It's unclear to me where and how "Review Issues" will be created. In general, anyone interested should be able to get notifications on whatever they want. And that's how MediaWiki itself already works.

@Aklapper The review issues do not exist yet, I was having a discussion in one of our meetings with @apaskulin, and she asked me a question : "If you could wave a magic wand and have any functionality in MediaWiki to support the review process, what would you want?". The idea was that I came up with a feature idea that could further improve documentation review processes, if the idea is good, it might be proposed and perhaps implemented. The "Proposing Review Issues" section is an answer to that question. It's not a perfect Idea yet but could use a lot of suggestions and feedbacks.

That would exist in the MediaWiki software by watching a talk page, and in Phabricator as tasks.

Oh, yeah! You could watch a talk page, or just decide watch a specific issue on that document.
Issues are quite similar to comments in Google Docs.

PS: General note: Please feel encouraged to replace "he" with "they" in sentences, for some more inclusiveness.

I am deeply sorry if that was perceived as offensive, that wasn't my intention. I will be more cautious with my use of pronouns.

I either don't understand, and/or we might be getting lost in terms here. One Phab project has one workboard. A workboard has columns. Columns can be used either for progress tracking or for categorizing tasks. I don't know what you mean by "the category the task belongs". Categories of tasks usually do not change.

My thoughts on how tasks should move through categories, and changing of tags has changed based on our discussion so far. So this task might need to be edited to reflect my new thoughts but basically the flow would look something like the steps below;

  • I create/update a document that needs a review.
  • I and my team internally do the "prototype and technical review", this means just the "non-technical review" is left.
  • Documents that need a review occupy a single column ("needs-review") on the documentation workboard.
  • I create a review task for the documentation and move it to the "needs-review" column on the workboard.
  • A reviewer in search documentations to review can simply go to the "needs-review" column to pick documentation and review.
  • When the review is done, the reviewer moves this task to the completed column.

What are your thoughts regarding these steps?

PS: General note: Please feel encouraged to replace "he" with "they" in sentences, for some more inclusiveness.

I am deeply sorry if that was perceived as offensive, that wasn't my intention. I will be more cautious with my use of pronouns.

It wasn't perceived as offensive; I just wanted to add a recommendation. No worries please! :)

@Aklapper The review issues do not exist yet, I was having a discussion in one of our meetings with @apaskulin, and she asked me a question : "If you could wave a magic wand and have any functionality in MediaWiki to support the review process, what would you want?".

Ah. :) That's good! <3 Thanks for the additional context that I was lacking!

My thoughts on how tasks should move through categories

Tasks should not move through categories. Tasks should move through workboard columns when workboard columns are used for progress tracking (instead of categorizing). See https://www.mediawiki.org/wiki/Phabricator/Project_management#Typical_examples_for_uses_of_workboards

  • I create/update a document that needs a review.
  • I and my team internally do the "prototype and technical review", this means just the "non-technical review" is left.
  • Documents that need a review occupy a single column ("needs-review") on the documentation workboard.
  • I create a review task for the documentation and move it to the "needs-review" column on the workboard.
  • A reviewer in search documentations to review can simply go to the "needs-review" column to pick documentation and review.
  • When the review is done, the reviewer moves this task to the completed column.

Basically yes when it comes to the concept of using workboards for progress tracking. :) Two things though:

The existing Documentation project workboard already uses workboard columns for task categorization.
The existing Documentation workboard does not and should not use workboard columns for tracking task progress or tracking task status (like "needs-review"), as workboard columns are already used for task categorization.
See "Multiple Boards" on https://www.mediawiki.org/wiki/Phabricator/Project_management#Implementing_Common_Project_Management_Practices_in_Phabricator for one potential way to have a workboard (not the workboard of Documentation though) for progress tracking.

And a very minor comment: Depending on the scope of a "review task" (basically: the criteria when to call that "review task" done); the last step ("reviewer moves this task to the completed column") might not be needed. When a task is done, the status field of a task should be changed from "open" to a closed status.

This is very insightful, thanks a lot @Aklapper.

Having a separate workboard for progress tracking is a wonderful Idea and would help organize things a lot better.

What are your thoughts on this;

  • A new workboard is created for progress tracking
  • The workboard has 2 columns, one contains tasks that needs a review("needs-review"), and the other contains tasks that have been reviewed or is currently being reviewed.
  • All new tasks goes to the "needs-review" column.
  • A task currently being reviewed goes to the other column
  • Once the review is done, that task is marked as resolved.

Makes sense to me if people™ for who this process is relevant can find out about that process (discoverability) and will spend time to follow that process. (Explicitly mentioning this because processes need social acceptance/buy-in, and the heavier a process the more likely people will ignore or circumvent it, generally speaking.)

Yes, I totally agree. This was why I suggested having a review admin who would oversee the review process and encourage people to adopt it. The admin would also be responsible for getting people to review these documentations, and since its just one review step, the process is not intense.

Makes sense to me if people™ for who this process is relevant can find out about that process (discoverability) and will spend time to follow that process.

As regards the discoverability, the go-to place for reviewers who want to review tasks would be the progress tracking board.

@srodlund @apaskulin, What do you think about the new review process proposed?

Gbahdeyboh renamed this task from GSoD 2020: Week 6 & 7 to GSoD 2020: Week 6 & 7 & 8.Nov 16 2020, 7:21 PM

As regards the discoverability, the go-to place for reviewers who want to review tasks would be the progress tracking board.

Right, and I wonder how people could find out, and which existing wiki pages or doc pages would have to document that and/or link to some board.
A process is only as good as its discoverability. :)

(On an unrelated note: I just watched https://www.youtube.com/watch?v=zm5Iw7jsyC4 (23 minutes) and that's a very interesting, great talk on managing docs.)

A process is only as good as its discoverability. :)

Very true! I plan on creating a separate document explaining this review process in-depth. The Technical Writer Guide, Technical Documentation Prioritization, and related documents will also be updated to summarize and link to the review process document.

I just watched https://www.youtube.com/watch?v=zm5Iw7jsyC4 (23 minutes) and that's a very interesting, great talk on managing docs.

Thanks for sharing this, I found it super helpful and interesting. It gives a very high level explanation of how they manage and review docs.
Having a similar structure at WMF might be difficult because documentation are wiki's and anyone could edit them without a review or approval before being published. Using Github as a review tool will not pose this same issue. This was why I came up with the "review issues" Idea, which might solve some of these problems to some reasonable extent.

Very true! I plan on creating a separate document explaining this review process in-depth.

I might very much derail here and go way too broad, plus the Documentation folks are very free to disagree:
I guess I'd prefer less pages to open and read in my web browser, and would love clearer scopes, audience separation, names for the existing three or four pages.
https://www.mediawiki.org/wiki/Documentation/Technical_writer_guide seems to only cover planning to write something completely new, from scratch, but its name does not really imply so. https://www.mediawiki.org/wiki/Documentation/New_technical_writers seems to be for totally new people in Wikimedia. For https://www.mediawiki.org/wiki/Documentation/Technical_documentation_prioritization it's unclear to me who is the audience of that page, or why that page lists locations, or why I should read it. And when going to https://www.mediawiki.org/wiki/Documentation I'm wondering which of all those links to read.

I think @apaskulin and @srodlund will be in the best position to share more context regarding these pages.

@Aklapper, the process looks good to Alex and Sarah as well. But we'll like to have a test review using this process just to be sure nothing was left out before proceeding to making it a standard review process.

For this, I would be needing a "Documentation Review" board, which I don't have enough privileges to create on phabricator. Can you please help with that?

Gbahdeyboh renamed this task from GSoD 2020: Week 6 & 7 & 8 to GSoD 2020: Week 6, 7, 8, 9 & 10 .Nov 25 2020, 9:43 AM
Gbahdeyboh renamed this task from GSoD 2020: Week 6, 7, 8, 9 & 10 to GSoD 2020: Week 6, 7, 8, & 9 .
Gbahdeyboh updated the task description. (Show Details)

(Wondering what exactly is left in this task, until it could be closed as resolved?)

Thanks for the prompt @Aklapper. Nothing is left, hence closing as resolved.

I plan on creating a separate document explaining this review process in-depth.

Out of curiosity, did this happen? If yes, then a link is welcome.