Pre-deployment QA: "Add a link"
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• ppelberg
	Mar 27 2021, 1:31 AM

Description

This is a QA task for finding additional bugs and edge cases in the Add a link structured task the Growth-Team team is preparing to deploy. @Etonkovidova will be testing that the functionality meets the specifications. But because the feature is built into the Visual Editor's interface, the Growth team is requesting help from @Ryasmeen to identify issues and bugs that require deep expertise in Visual Editor to find.

Testing instructions

Environments

Wiki(s) to test on: beta wikis of Arabic, Vietnamese, English
Platform(s) to test on: desktop web, iOS Safari, Android Chrome.
Page(s) to test on: any articles with link suggestions from the newcomer homepage.

Instructions
In each of the "Environments" listed above, proceed through the link suggestion workflow, looking for...

...ways that the workflow's experience can break for the user.
...situations where the edit saved by the workflow causes unexpected changes not intended by the workflow (e.g. link applied to wrong word, link with wrong destination, any corruption in result).

Done

Test instructions are completed and any unexpected behavior that surfaces in the process is documented in Phabricator.

Test results

Cannot switch to edit source mode, while trying to switch to Source mode either from the menu link "Edit Source" or the switcher inside VE. There is an error in the console, if you try to switch after applying a link:

Screen Shot 2021-04-28 at 12.59.36 PM.png (1×2 px, 683 KB)

Follow-up: Will be fixed in T269653: Add a link: edit mode toggle (machine suggestions & visual). FWIW the error comes from the Jade extension, it's unclear to me why Jade is creating a diff widget for this scenario, but it seems like a bug in that code

Cannot navigate to History page to check diff until I have gone through all the link suggestions on that page, is that expected?

Follow-up: Yes, expected. Will also be addressed in T269653: Add a link: edit mode toggle (machine suggestions & visual) / T280422: Link recommendation inspector - reloading page workflows

When you apply a link and on the second attempt if you then select "No", it does not remove the previously applied link.

Follow-up: Tracked in T281462: Prevent link suggestions from appearing again after publishing an edit / T269659: Add a link: post-edit dialog

The visual diff showed two entires for one link change and some changes to reflist that are not relevant. I forgot to take a screenshot for that. But for another instance the diff was empty after changing some links.

Screen Shot 2021-04-28 at 11.36.14 AM.png (966×1 px, 331 KB)

Follow-up: Tracked in T280980: Review changes button does not show diff in Visual mode

The diff for applying links look different from the diff that you get when you add a link in usual way. Is that expected?

The diff with Add a link feature:

Screen Shot 2021-04-28 at 11.46.41 AM.png (1×1 px, 499 KB)

The diff with article editing mode:

Screen Shot 2021-04-28 at 11.48.22 AM.png (558×1 px, 88 KB)

Follow-up: Tracked in T280980: Review changes button does not show diff in Visual mode

The shortcut key for VisualEditor to open up link inspector also brings up VE's own link inspector resulting into a situation like this:

Screen Shot 2021-04-28 at 2.21.19 PM.png (1×1 px, 510 KB)

You can't apply the link though, I guess that's expected?

Follow-up: Tracked in T281434: Add link: VE shortcut opens a regular link inspector overlapping add link context item

In fact, you can open all the inspectors/dialogs in this mode for different elements on the article page but can't really make any other edit. Not sure if that's expected?

Follow-up: Tracked in T280129: Disable interactions with all context items except for RecommendedLinkContextItem

How do I get back to normal editing mode? It seems I can't do anything else Until I go through all link suggestions for that article.

Follow-up: Tracked in T269653: Add a link: edit mode toggle (machine suggestions & visual)

We have another shortcut key for opening the Find and Replace dialog. Using that will auto fill the Find and Replace dialog with the next link suggestion text and from that point, the link suggestion context menu does not open anymore for that article. The only option to get out of that limbo state is to click on "Read" again.

Follow-up: Tracked in T281434: Add link: VE shortcut opens a regular link inspector overlapping add link context item

Also, at some point I got two link suggestion cartouches. I don't have the exact steps to reproduce this yet.

Screen Shot 2021-04-28 at 1.17.21 PM.png (1×1 px, 530 KB)

Follow-up: Tracked in T267690: Add a link in VE: don't write to or read from restored edits, I wonder if the patch for this didn't make it into beta labs before QA testing happened.

The link suggestion context also opened up at the top left corner of the page and clicking next was just showing up the next suggestion but was not pointing at the right text position.

Screen Shot 2021-04-28 at 2.47.51 PM.png (980×1 px, 342 KB)

Follow-up: Tracked in T281464: Wrong positioning for link suggestion context item

Unlike other dialogs, where it seems all the buttons were intentionally hidden, the "Add a citation" button on the "Citation needed" template context menu was visible and clicking on it makes a weird transition animation but of-course because of suppressing the actual action, the dialog to Add a citation does not open.

Screen Shot 2021-04-28 at 6.41.26 PM.png (1×1 px, 530 KB)

Follow-up: Tracked in T281434: Add link: VE shortcut opens a regular link inspector overlapping add link context item

Made an edit on an article in source mode and then switched to VisualEditor, was presented with the Add a link workflow but nothing happened after clicking on "Publish changes". There was no error message on the console.

Follow-up: Tracked in T269653: Add a link: edit mode toggle (machine suggestions & visual)

The blue-highlight for different elements present on the article page can overlap with the adjacent link suggestion context menu.

Follow-up: Tracked in T281463: Set background color to recommended link context item

Related Objects

Mentioned In: T281464: Wrong positioning for link suggestion context item
T281463: Set background color to recommended link context item
T281462: Prevent link suggestions from appearing again after publishing an edit
Mentioned Here: T282899: [wmf.5] Add link: Unresponsive controls when context item overlaps templates
T281464: Wrong positioning for link suggestion context item
T281463: Set background color to recommended link context item
T269659: Add a link: post-edit dialog
T281462: Prevent link suggestions from appearing again after publishing an edit
T267690: Add a link in VE: don't write to or read from restored edits
T269653: Add a link: edit mode toggle (machine suggestions & visual)
T280129: Disable interactions with all context items except for RecommendedLinkContextItem
T280980: Review changes button does not show diff in Visual mode
T281434: Add link: VE shortcut opens a regular link inspector overlapping add link context item
T280422: Link recommendation inspector - reloading page workflows
T267706: Add a link in VE: Use a permanent context instead of VE's built-in context

Event Timeline

• ppelberg created this task.Mar 27 2021, 1:31 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 27 2021, 1:31 AM

@MMiller_WMF I'm assigning this task over to you to add details to the task description's ===Testing instructions section. When done, please ping me.

This page GrowthExperiments#Add_link_(AI)_structured_task has some useful information on how to start testing

enabling Homepage
where to test
links to documentation on Add link Structured task and to mockups (Desktop and mobile)

@ppelberg -- I've added detail to the task description. Please let me know if anything else is needed. We're currently planning for this review to begin April 13. Thank you and @Ryasmeen for helping with our team's project!

In T278600#6968529, @MMiller_WMF wrote:

@ppelberg -- I've added detail to the task description. Please let me know if anything else is needed. We're currently planning for this review to begin April 13. Thank you and @Ryasmeen for helping with our team's project!

Thank you, @MMiller_WMF – what you've added looks good to @Ryasmeen and me. Should Rummana have questions as she starts testing, she will comment them here.

And per the conversation you and I had offline today, we will plan for work on this task to start on the 26-April.

kostajh moved this task from Backlog to April 26 - April 30 on the Add-Link board.Apr 8 2021, 8:30 PM

MBinder_WMF added a project: Growth-Team-Filtering.Apr 15 2021, 6:47 PM

kostajh triaged this task as Medium priority.Apr 20 2021, 9:19 AM

kostajh moved this task from Incoming to QA on the Growth-Team (Sprint 0 (Growth Team)) board.

• Rileych removed a project: Growth-Team-Filtering.Apr 20 2021, 4:44 PM

Hi @Ryasmeen! We're ready for you to test out "add a link" this week. Like it says in the task description, because our feature hacks the Visual Editor, we would like you to find the issues that are hard to find, that require your specialized knowledge of the visual editor. Rather than validating that our workflow satisfies every requirement, we just want you to go through the workflow on desktop and mobile, and try to break it.

Not all our styling and aesthetics are in place, but the functionality should be.

On the beta wikis, to start the workflow...

Enable the homepage and help panel features in your preferences. The homepage preferences are at the bottom of "User profile" and the help panel preferences is in "Editing".
Go to your homepage by clicking/tapping your username.
Select the "Add links between articles" tasks for your suggested edits feed using the "Easy edits" filter.
Choose an article and get started.

Please reply in a comment with any issues you see!

In T278600#7032387, @MMiller_WMF wrote:

Hi @Ryasmeen! We're ready for you to test out "add a link" this week. Like it says in the task description, because our feature hacks the Visual Editor, we would like you to find the issues that are hard to find, that require your specialized knowledge of the visual editor. Rather than validating that our workflow satisfies every requirement, we just want you to go through the workflow on desktop and mobile, and try to break it.

Not all our styling and aesthetics are in place, but the functionality should be.

On the beta wikis, to start the workflow...

Enable the homepage and help panel features in your preferences. The homepage preferences are at the bottom of "User profile" and the help panel preferences is in "Editing".

Go to your homepage by clicking/tapping your username.

Select the "Add links between articles" tasks for your suggested edits feed using the "Easy edits" filter.

Choose an article and get started.

Please reply in a comment with any issues you see!

@Ryasmeen (cc @MMiller_WMF) I apologize for the late notice here, but because we made faster progress than expected on implementing a rearchitecting of the VisualEditor integration code (T267706: Add a link in VE: Use a permanent context instead of VE's built-in context) and are therefore close to merging that, we would like for you to wait for that to be merged and deployed on beta before doing the QA steps in this task. We think that it could be possible for you to begin QA on Wednesday, but I will comment here after that patch is merged.

In T278600#7033111, @kostajh wrote:

In T278600#7032387, @MMiller_WMF wrote:

Hi @Ryasmeen! We're ready for you to test out "add a link" this week. Like it says in the task description, because our feature hacks the Visual Editor, we would like you to find the issues that are hard to find, that require your specialized knowledge of the visual editor. Rather than validating that our workflow satisfies every requirement, we just want you to go through the workflow on desktop and mobile, and try to break it.

Not all our styling and aesthetics are in place, but the functionality should be.

On the beta wikis, to start the workflow...

Enable the homepage and help panel features in your preferences. The homepage preferences are at the bottom of "User profile" and the help panel preferences is in "Editing".

Go to your homepage by clicking/tapping your username.

Select the "Add links between articles" tasks for your suggested edits feed using the "Easy edits" filter.

Choose an article and get started.

Please reply in a comment with any issues you see!

@Ryasmeen (cc @MMiller_WMF) I apologize for the late notice here, but because we made faster progress than expected on implementing a rearchitecting of the VisualEditor integration code (T267706: Add a link in VE: Use a permanent context instead of VE's built-in context) and are therefore close to merging that, we would like for you to wait for that to be merged and deployed on beta before doing the QA steps in this task. We think that it could be possible for you to begin QA on Wednesday, but I will comment here after that patch is merged.

Sounds good @kostajh.

@Ryasmeen The patches in T267706: Add a link in VE: Use a permanent context instead of VE's built-in context have been merged and should be on betalabs shortly, so this task is ready to go.

Ryasmeen added a project: Editing QA.Apr 28 2021, 5:48 PM

In T278600#7032387, @MMiller_WMF wrote:

Hi @Ryasmeen! We're ready for you to test out "add a link" this week. Like it says in the task description, because our feature hacks the Visual Editor, we would like you to find the issues that are hard to find, that require your specialized knowledge of the visual editor. Rather than validating that our workflow satisfies every requirement, we just want you to go through the workflow on desktop and mobile, and try to break it.

Not all our styling and aesthetics are in place, but the functionality should be.

On the beta wikis, to start the workflow...

Enable the homepage and help panel features in your preferences. The homepage preferences are at the bottom of "User profile" and the help panel preferences is in "Editing".

Go to your homepage by clicking/tapping your username.

Select the "Add links between articles" tasks for your suggested edits feed using the "Easy edits" filter.

Choose an article and get started.

Please reply in a comment with any issues you see!

@MMiller_WMF : I have started testing this on English beta wiki and finding a lot of issues. There is a lot to cover :) However, I would like to point out that, it might not be possible for me to conceptually separate the feature workflow testing since I am doing a full integration testing covering different areas. So there might be some issues that are maybe more relevant to the workflow and redundant reporting. I will mention all of them here and leave it up to you to sort them out :)

Issues found so far:

Cannot switch to edit source mode, while trying to switch to Source mode either from the menu link "Edit Source" or the switcher inside VE. There is an error in the console, if you try to switch after applying a link:

Cannot navigate to History page to check diff until I have gone through all the link suggestions on that page, is that expected?

When you apply a link and on the second attempt if you then select "No", it does not remove the previously applied link.

The visual diff showed two entires for one link change and some changes to reflist that are not relevant. I forgot to take a screenshot for that. But for another instance the diff was empty after changing some links.

The diff for applying links look different from the diff that you get when you add a link in usual way. Is that expected?

The diff with Add a link feature:

The diff with article editing mode:

The shortcut key for VisualEditor to open up link inspector also brings up VE's own link inspector resulting into a situation like this:

You can't apply the link though, I guess that's expected?

In fact, you can open all the inspectors/dialogs in this mode for different elements on the article page but can't really make any other edit. Not sure if that's expected?

How do I get back to normal editing mode? It seems I can't do anything else Until I go through all link suggestions for that article.

We have another shortcut key for opening the Find and Replace dialog. Using that will auto fill the Find and Replace dialog with the next link suggestion text and from that point, the link suggestion context menu does not open anymore for that article. The only option to get out of that limbo state is to click on "Read" again.

Also, at some point I got two link suggestion cartouches. I don't have the exact steps to reproduce this yet.

The link suggestion context also opened up at the top left corner of the page and clicking next was just showing up the next suggestion but was not pointing at the right text position.

Let me know if you need more details on any of these. I will continue testing this and there are some more issues relevant to the workflow, I can post them here later if you think that's helpful! :)

Thanks, @Ryasmeen!
(1) - (2) - are part of T280422: Link recommendation inspector - reloading page workflows

When you apply a link and on the second attempt if you then select "No", it does not remove the previously applied link.

Do you mean that "No" button doesn't act as a toggle?

(4)-(5) - is still work in process

In fact, you can open all the inspectors/dialogs in this mode for different elements on the article page but can't really make any other edit. Not sure if that's expected?

Yes, it's expected.

How do I get back to normal editing mode? It seems I can't do anything else Until I go through all link suggestions for that article.

Presently, editing is possible in the Source editing. The idea is that a user would use Add link machine suggestion mode without an option to use VE.

We have another shortcut key for opening the Find and Replace dialog. Using that will auto fill the Find and Replace dialog with the next link suggestion text and from that point, the link suggestion context menu does not open anymore for that article. The only option to get out of that limbo state is to click on "Read" again.

Yes, using shortcuts - cmd+F, cmd+G, cmd+Up arrow+G - will disable the link inspector. The page needs to be reloaded to bring the link inspector back.

Also, at some point I got two link suggestion cartouches. I don't have the exact steps to reproduce this yet.

Yes, it's a known issue.

The link suggestion context also opened up at the top left corner of the page and clicking next was just showing up the next suggestion but was not pointing at the right text position.

Hmm, that issue I've never seen. Although I did see it with other ooui elements in the past. do you have steps to reproduce it?

@Ryasmeen -- thank you so much for doing this and for leaving us notes along the way. While some of the issues you're listing are known, I think most of them are not, so I think it's great that we're getting your thoughts. I think it was definitely smart to look at the diff -- I hadn't thought of doing that.

@Etonkovidova -- could you please start breaking off @Ryasmeen's findings into separate Phabricator tasks when appropriate?

I just tested cmd+up arrow +k - it does open the link inspector which would overlaps the context item inspector.

The shortcut key for VisualEditor to open up link inspector also brings up VE's own link inspector resulting into a situation like this:

Filed as T281434: Add link: VE shortcut opens a regular link inspector overlapping add link context item

@Etonkovidova :

When you apply a link and on the second attempt if you then select "No", it does not remove the previously applied link.

Do you mean that "No" button doesn't act as a toggle?

Not exactly. It actually does act like a toggle on the same session, as in when you click on it again on the same session it sets the status of that suggestion to unchanged which is denoted by "-" on the save dialog it seems. The issue I am describing here happens when you already applied a link and submitted your changes, then re-opened the article again and going through the same link suggestion again. If at that point, you click "No" on the link suggestion dialog, it does not remove the actual link whereas on the save dialog it shows 'x" for it. Does that make sense? :) Also, noticed that it doesn't remember the previous state of the buttons on the link suggestion dialog in the next edit session.

The link suggestion context also opened up at the top left corner of the page and clicking next was just showing up the next suggestion but was not pointing at the right text position.

Hmm, that issue I've never seen. Although I did see it with other ooui elements in the past. do you have steps to reproduce it?

Yeah, I remember seeing that too! :) I do not have the steps to reproduce this at the moment unfortunately.

In T278600#7044172, @Ryasmeen wrote:

@Etonkovidova :

When you apply a link and on the second attempt if you then select "No", it does not remove the previously applied link.

Do you mean that "No" button doesn't act as a toggle?

Not exactly. It actually does act like a toggle on the same session, as in when you click on it again on the same session it sets the status of that suggestion to unchanged which is denoted by "-" on the save dialog it seems. The issue I am describing here happens when you already applied a link and submitted your changes, then re-opened the article again and going through the same link suggestion again. If at that point, you click "No" on the link suggestion dialog it does not remove the actual link whereas on the save dialog it shows 'x" for it. Does that make sense? :) Also, noticed that it doesn't remember the previous status of the buttons on the link suggestion dialog in the next edit session.

Thanks, for the details! I'll look into it.

Few more issues:

Unlike other dialogs, where it seems all the buttons were intentionally hidden, the "Add a citation" button on the "Citation needed" template context menu was visible and clicking on it makes a weird transition animation but of-course because of suppressing the actual action, the dialog to Add a citation does not open.

Made an edit on an article in source mode and then switched to VisualEditor, was presented with the Add a link workflow but nothing happened after clicking on "Publish changes". There was no error message on the console.

The blue-highlight for different elements present on the article page can overlap with the adjacent link suggestion context menu.

kostajh updated the task description. (Show Details)Apr 29 2021, 10:14 AM

@Ryasmeen thank you for your thorough testing! I've updated the task description and referenced the relevant phab tasks (some existing, some new) that track the issues you've noted.

kostajh updated the task description. (Show Details)Apr 29 2021, 10:37 AM

kostajh mentioned this in T281462: Prevent link suggestions from appearing again after publishing an edit.Apr 29 2021, 10:42 AM

kostajh updated the task description. (Show Details)

kostajh mentioned this in T281463: Set background color to recommended link context item.Apr 29 2021, 10:44 AM

kostajh updated the task description. (Show Details)

kostajh mentioned this in T281464: Wrong positioning for link suggestion context item.

kostajh updated the task description. (Show Details)Apr 29 2021, 10:47 AM

• ppelberg edited projects, added Editing-team (Tracking); removed Editing QA, Editing-team.Apr 29 2021, 7:20 PM

• ppelberg moved this task from Backlog to Dependent on others on the Editing-team (Tracking) board.

kostajh moved this task from April 26 - April 30 to May 10 - May 14 on the Add-Link board.May 3 2021, 2:42 PM

kostajh moved this task from May 10 - May 14 to Done / QA on the Add-Link board.May 10 2021, 1:29 PM

Reviewed the task and re-checked the reported issues on testwiki wmf.6

Issue reported	Test result on `testwiki wmf.6`
✅ 1. Cannot switch to edit source mode, while trying to switch to Source mode either from the menu link "Edit Source" or the switcher inside VE. There is an error in the console, if you try to switch after applying a link.	The switch to "Edit source" is disabled.
✅ 2. Cannot navigate to History page to check diff until I have gone through all the link suggestions on that page	A user can switch to View history any time - without making interactions with the context item or after submitting few 'Yes'/''No' questions.
✅ 3. When you apply a link and on the second attempt if you then select "No", it does not remove the previously applied link.	Works as expected - T281462 T269659 are checked
✅ 4. The visual diff showed two entries for one link change and some changes to reflist that are not relevant. I forgot to take a screenshot for that. But for another instance the diff was empty after changing some links.	Works as expected
✅ 5. The diff for applying links look different from the diff that you get when you add a link in usual way.	Works as expected.
✅ 6. The shortcut key for VisualEditor to open up link inspector also brings up VE's own link inspector.	the shortcuts are disabled now
✅ 7. In fact, you can open all the inspectors/dialogs in this mode for different elements on the article page but can't really make any other edit.	inspectors/dialogs are disabled
✅ 8. How do I get back to normal editing mode? It seems I can't do anything else Until I go through all link suggestions for that article.	only one 'yes' or 'no' can be published and a user is returned to normal Edit mode; some improvements will be done in T269653: Add a link: edit mode toggle (machine suggestions & visual)
✅ 9. We have another shortcut key for opening the Find and Replace dialog. Using that will auto fill the Find and Replace dialog with the next link suggestion text and from that point, the link suggestion context menu does not open anymore for that article. The only option to get out of that limbo state is to click on "Read" again.	All shortcuts are disabled
✅ 10. Also, at some point I got two link suggestion cartouches. I don't have the exact steps to reproduce this yet.	Checked - the issue is not present anymore
✅ 11. The link suggestion context also opened up at the top left corner of the page and clicking next was just showing up the next suggestion but was not pointing at the right text position.	the follow up task is marked as Resolved T281464
✅ 12. Unlike other dialogs, where it seems all the buttons were intentionally hidden, the "Add a citation" button on the "Citation needed" template context menu was visible and clicking on it makes a weird transition animation but of-course because of suppressing the actual action, the dialog to Add a citation does not open.	all VE actions are disabled
✅ 13. Made an edit on an article in source mode and then switched to VisualEditor, was presented with the Add a link workflow but nothing happened after clicking on "Publish changes". There was no error message on the console.	Not possible to switch between editing modes now.
✅ 14. The blue-highlight for different elements present on the article page can overlap with the adjacent link suggestion context menu.	the blue background color is not displayed anymore; there is an additional issue filed as T282899: [wmf.5] Add link: Unresponsive controls when context item overlaps templates

Thanks @Ryasmeen again for the excellent QA analysis!

Restricted Application added a project: User-Ryasmeen. · View Herald TranscriptMay 19 2021, 2:56 AM

	F34432944: Screen Shot 2021-04-28 at 6.41.26 PM.png
	Apr 29 2021, 2:11 AM

	F34432692: Screen Shot 2021-04-28 at 2.47.51 PM.png
	Apr 28 2021, 10:00 PM

	F34432625: Screen Shot 2021-04-28 at 11.48.22 AM.png
	Apr 28 2021, 10:00 PM

	F34432628: Screen Shot 2021-04-28 at 2.21.19 PM.png
	Apr 28 2021, 10:00 PM

	F34432689: Screen Shot 2021-04-28 at 1.17.21 PM.png
	Apr 28 2021, 10:00 PM

	F34432540: Screen Shot 2021-04-28 at 12.59.36 PM.png
	Apr 28 2021, 10:00 PM

Pre-deployment QA: "Add a link"Closed, ResolvedPublicActions

Description

Testing instructions

Done

Test results

Related Objects

Event Timeline

Pre-deployment QA: "Add a link"
Closed, ResolvedPublic
Actions