hi @Dbrant - can the updated APK for testing be linked to this ticket when ready so that @dchen and @schoenbaechler can continue pilot and test planning? Thanks!

Hey @RHo, just talked to the team and @Dbrant mentioned that we’re going to use the Alpha for it. @Sharvaniharan is doing the last few tweaks and if @ABorbaWMF doesn’t have the time I’ll QA the app according to @dchen’s protocol before Monday. Does that sound fine to you?

In T211109#4873483, @schoenbaechler wrote:

Hey @RHo, just talked to the team and @Dbrant mentioned that we’re going to use the Alpha for it. @Sharvaniharan is doing the last few tweaks and if @ABorbaWMF doesn’t have the time I’ll QA the app according to @dchen’s protocol before Monday. Does that sound fine to you?

• Sure, I'd just request the specific Alpha version is clearly marked and uploaded for testing across all participants.

Copy pasting the email I just sent:

Hi Sharvani, thanks for the update! Looping in @Daisy, @Dmitry, @Rita and @Carolyn to this email thread.

I followed Daisy’s protocol and found the following discrepancies in version 2.7.269-alpha-2019-01-12:

Users need to be logged in to their accounts in the current alpha to unlock the “App editor tasks“ menu item and dialogs. This is currently not mentioned in the protocol.

• When setting multiple languages in the app’s onboarding after initial launch as described in T207334, the second “Title description“ edit triggers the Translating tile descriptions editing has been unlocked dialog. If no additional languages are set in the app’s onboarding (or settings), the “More tasks for multilingual editors“ item in “App editor tasks“ can’t be accessed at all (screen). Plus, it wasn’t completely clear to me when and why sometimes the “More tasks for multilingual editors“ appears visually locked (which is sometimes accessible in locked state, sometimes not) and when unlocked. Ideally, it would always be in unlocked state for the usability testing to not cause confusion and to make it easier for participants to “Add languages“ if they haven’t done it already.

• A edit history/app contributions screen does not exist in the alpha yet. The current indicators that users have in the alpha are on the main view in “App editor tasks" where the interface communicates how many “in-app contributions“ they’ve made thus far and within “Translate title descriptions" or “Add title descriptions“. In the latter, they first need to tap the three dots (“More“) at the top right, then “My contributions“. I think this is sufficient for the usability tests since if users are tapping on either of those indicators then they’d successfully identified the place to find it.

• In regards to “Oh no - you think you made a mistake (…)“: Going back by swiping right works, but it currently doesn’t update the visual output of the card with the edit that has been made previously. However, it’s saved to the database; so if users navigate out of “App Editor Tasks“ and e.g. manually search for that previously edited article, the title description includes the edit they’ve just made. We can still observe if users are actually going back by swiping right or not.

Also added these comments to Daisy’s Edit Action Android Feed Planning Protocol/Survey doc and to the corresponding Phab ticket (
T211109). IMO the things listed are not necessarily blockers for the test, we just need to be aware of it. What do you think @Rita and/or @Carolyn?

@Daisy: I scheduled another pilot session for this Monday, January 14, 10:30 am your time (PT), let me know if that works for you.

Thanks,
Robin

Hi Robin , thanks for going through the proto again, just have a couple comments inline below.

In T211109#4875777, @schoenbaechler wrote:

Copy pasting the email I just sent:

Hi Sharvani, thanks for the update! Looping in @Daisy, @Dmitry, @Rita and @Carolyn to this email thread.

I followed Daisy’s protocol and found the following discrepancies in version 2.7.269-alpha-2019-01-12:

Users need to be logged in to their accounts in the current alpha to unlock the “App editor tasks“ menu item and dialogs. This is currently not mentioned in the protocol.

• Yes, this is correct, since the prerequisite is that the user is an existing logged in user of the title description feature.

• When setting multiple languages in the app’s onboarding after initial launch as described in T207334, the second “Title description“ edit triggers the Translating tile descriptions editing has been unlocked dialog. If no additional languages are set in the app’s onboarding (or settings), the “More tasks for multilingual editors“ item in “App editor tasks“ can’t be accessed at all (screen). Plus, it wasn’t completely clear to me when and why sometimes the “More tasks for multilingual editors“ appears visually locked (which is sometimes accessible in locked state, sometimes not) and when unlocked. Ideally, it would always be in unlocked state for the usability testing to not cause confusion and to make it easier for participants to “Add languages“ if they haven’t done it already.

1. The title of this task "More tasks for multilingual editors" is misleading since it makes it sound like tapping on this takes you to a bunch of different translation tasks. Let's revert to the title "Translate title descriptions" copy
2. I thought we had decided for test purposes only it will initially require 1 title description addition via the task queue to unlock translate title descriptions? The copy about 50 title descriptions should therefore be updated.
3. Since we are testing multilingual people so they can complete the full test, it may make things easier to build in a check that the app should already have more than one language set.

• A edit history/app contributions screen does not exist in the alpha yet. The current indicators that users have in the alpha are on the main view in “App editor tasks" where the interface communicates how many “in-app contributions“ they’ve made thus far and within “Translate title descriptions" or “Add title descriptions“. In the latter, they first need to tap the three dots (“More“) at the top right, then “My contributions“. I think this is sufficient for the usability tests since if users are tapping on either of those indicators then they’d successfully identified the place to find it.

• in the protocol doc comments, we had discussed adding a mock of the contributions screen so that users can describe their understanding and talk about how useful they find the information shown in this screen (would they go to check this often? Etc), beyond just knowing where to access it. However, defer to Daisy and yourself to determine how much effort vs reward this is to build into the task list versus showing them a mock afterwards.

• In regards to “Oh no - you think you made a mistake (…)“: Going back by swiping right works, but it currently doesn’t update the visual output of the card with the edit that has been made previously. However, it’s saved to the database; so if users navigate out of “App Editor Tasks“ and e.g. manually search for that previously edited article, the title description includes the edit they’ve just made. We can still observe if users are actually going back by swiping right or not.

Also added these comments to Daisy’s Edit Action Android Feed Planning Protocol/Survey doc and to the corresponding Phab ticket (
T211109). IMO the things listed are not necessarily blockers for the test, we just need to be aware of it. What do you think @Rita and/or @Carolyn?

@Daisy: I scheduled another pilot session for this Monday, January 14, 10:30 am your time (PT), let me know if that works for you.

Thanks,
Robin

Thanks for your feedback @RHo!

1. The title of this task "More tasks for multilingual editors" is misleading since it makes it sound like tapping on this takes you to a bunch of different translation tasks. Let's revert to the title "Translate title descriptions" copy
2. I thought we had decided for test purposes only it will initially require 1 title description addition via the task queue to unlock translate title descriptions? The copy about 50 title descriptions should therefore be updated.

Agreed and here are my copy suggestions for the translation task’s states in the Alpha and for usability testing:

User hasn’t set multiple languages in the app’s settings:
Title: Translate title descriptions
Description: Translation tasks are available if you read and write in more than one Wikipedia language.
Actions: Not for me | Add languages

Locked and user has set multiple languages in the app’s settings:
Title: Translate title descriptions
Description: Translate title descriptions into other languages.
Unlock hint: Locked until you’ve added 2 title descriptions.

Unlocked and user has set multiple languages in the app’s settings:
Title: Translate title descriptions
Description: Translate title descriptions into other languages.

@Sharvaniharan, could you please update the copy accordingly? Thanks.

Hi & thanks for your comments @Charlotte, adding them to this task as well.

Hey Robin,

Thanks for this! Comments inline:

Hi Sharvani, thanks for the update! Looping in @Daisy, @Dmitry, @Rita and @Carolyn to this email thread. 

I followed Daisy's protocol and found the following discrepancies in version 2.7.269-alpha-2019-01-12:

• Users need to be logged in to their accounts in the current alpha to unlock the "App editor tasks" menu item and dialogs. This is currently not mentioned in the protocol.

This is expected behaviour and will need to be added to the protocol as per our email conversation last week.

• When setting multiple languages in the app's onboarding after initial launch as described in T207334, the second "Title description" edit triggers the Translating tile descriptions editing has been unlocked dialog. If no additional languages are set in the app's onboarding (or settings), the "More tasks for multilingual editors" item in "App editor tasks" can't be accessed at all (screen).\

Plus, it wasn't completely clear to me when and why sometimes the "More tasks for multilingual editors" appears visually locked (which is sometimes accessible in locked state, sometimes not) and when unlocked. Ideally, it would always be in unlocked state for the usability testing to not cause confusion and to make it easier for participants to "Add languages" if they haven't done it already.

Yes, this is unexpected, and something we definitely need to work out for the final version.

• A edit history/app contributions screen does not exist in the alpha yet. The current indicators that users have in the alpha are on the main view in "App editor tasks" where the interface communicates how many "in-app contributions" they've made thus far and within "Translate title descriptions" or "Add title descriptions". In the latter, they first need to tap the three dots ("More") at the top right, then "My contributions". I think this is sufficient for the usability tests since if users are tapping on either of those indicators then they'd successfully identified the place to find it.\

I think you're right that for the test that it's enough to see whether they can find where it would be.

• In regards to "Oh no - you think you made a mistake (...)": Going back by swiping right works, but it currently doesn't update the visual output of the card with the edit that has been made previously. However, it's saved to the database; so if users navigate out of "App Editor Tasks" and e.g. manually search for that previously edited article, the title description includes the edit they've just made. We can still observe if users are actually going back by swiping right or not.\

Also added these comments to Daisy's Edit Action Android Feed Planning Protocol/Survey doc and to the corresponding Phab ticket (

T211109). IMO the things listed are not necessarily blockers for the test, we just need to be aware of it. What do you think @Rita and/or @Carolyn?

I would tend to agree. Thanks for going through the alpha so thoroughly and updating the tickets, Robin. Really appreciated!

@schoenbaechler not much for me to add here, but let me know if you need anything specific for a mock-up of the score card.

• When setting multiple languages in the app's onboarding after initial launch as described in T207334, the second "Title description" edit triggers the Translating tile descriptions editing has been unlocked dialog. If no additional languages are set in the app's onboarding (or settings), the "More tasks for multilingual editors" item in "App editor tasks" can't be accessed at all (screen).\

Plus, it wasn't completely clear to me when and why sometimes the "More tasks for multilingual editors" appears visually locked (which is sometimes accessible in locked state, sometimes not) and when unlocked. Ideally, it would always be in unlocked state for the usability testing to not cause confusion and to make it easier for participants to "Add languages" if they haven't done it already.

Yes, this is unexpected, and something we definitely need to work out for the final version.

@schoenbaechler @Charlotte [For PROTOTYPE only] I need a few more clarifications on how this should behave. I did remember that we decided in th estand-up that the "More tasks for multilingual editors" will always be unlocked.. but then I did not know when the translation unlock dialog should be showed, and what it would actually unlock? Could you please let me know the flow :
after onboarding and unlocking the edit tasks feed,

1. Both title desc and translate desc tasks are unlocked?
2. If yes, do we need the translation unlocked message?
3. If no, then should it have the general disabled look like the caption tasks, or should their be change in ui/functionality?
4. You unlock title desc, you make a valid edit, you see the translation unlocked message and hit 'Get started' and go to translation edit screen, should back button land you in title desc screen or edit feed screen?

User hasn’t set multiple languages in the app’s settings:
Title: Translate title descriptions
Description: Translation tasks are available if you read and write in more than one Wikipedia language.
Actions: Not for me | Add languages

Just following up from our discussion @schoenbaechler , This logic will not be updated for the prototype, since we are already asking the user to add languages during onboarding. Also because i would have to add special logic to have a locked task with actionable buttons.

2. For the use case "You unlock title desc, you make a valid edit, you see the translation unlocked message and hit 'Get started' and go to translation edit screen, should back button land you in title desc screen or edit feed screen?"

We decided that the current behavior is ok, that the user will go back to the title description screen.

3. Logic has been added so that the toc tooltip doesn't clash with the first unlock dialog - the edit feed unlock dialog. However, further down the road, when they are trying to add descriptions or translate descriptions, if the users tap on the article to read before editing, the tooltip might show up [based on toolti logic].

We decided that is fine.

Thanks @Sharvaniharan, adding this for the sake of completion:

• Currently, the translate title descriptions dialog is triggered after every title description edit that’s made after the second one. It should only be triggered once after the second one (“Locked until you’ve added 2 title descriptions.)

Change 484310 had a related patch set uploaded (by Sharvaniharan; owner: Sharvaniharan):
[apps/android/wikipedia@master] Translation task updates

https://gerrit.wikimedia.org/r/484310

Apologies to be jumping in without having attended the stand, but since it's really late in Europe I thought I'd try to answer some questions:

In T211109#4878296, @Sharvaniharan wrote:

@schoenbaechler @Charlotte [For PROTOTYPE only] I need a few more clarifications on how this should behave. I did remember that we decided in th estand-up that the "More tasks for multilingual editors" will always be unlocked.. but then I did not know when the translation unlock dialog should be showed, and what it would actually unlock? Could you please let me know the flow :
after onboarding and unlocking the edit tasks feed,

1. Both title desc and translate desc tasks are unlocked?

My understanding is that Title Description task are unlocked after the user makes one Title Description edit from their feed.

If I also understood correctly, the user must tap to through the 'unlock translation tasks' card so that the on boarding messaging can be tested, but they will not need to go through the language CAPTCHA

2. If yes, do we need the translation unlocked message?

My gut instinct based on the protocol is yes, because of step 2 in the 'Translate title description' section of the protocol.

4. You unlock title desc, you make a valid edit, you see the translation unlocked message and hit 'Get started' and go to translation edit screen, should back button land you in title desc screen or edit feed screen?

My guess is back to the edit feed screen as Title Description Translation is not nested within Title Description

@schoenbaechler does this seem correct?

@cmadeo Thank you for the suggestions. I was actually able to have a chat with @schoenbaechler before he left for the day and he was able to clarify. Other than the last point , the back button functionality... rest were the same. Will re-confirm with him tomorrow.

Change 484310 merged by jenkins-bot:
[apps/android/wikipedia@master] Translation task updates

https://gerrit.wikimedia.org/r/484310

Change 484340 had a related patch set uploaded (by Sharvaniharan; owner: Sharvaniharan):
[apps/android/wikipedia@master] Fix: Translation unlock should not show more than once

https://gerrit.wikimedia.org/r/484340

Change 484340 merged by jenkins-bot:
[apps/android/wikipedia@master] Fix: Translation unlock should not show more than once

https://gerrit.wikimedia.org/r/484340

@schoenbaechler @RHo @dchen @Charlotte final apk link has been updated.

Hey @Sharvaniharan, I couldn’t install the apk with the link you provided on my phone. Asked @RHo to try it as well and she wasn’t able to do so either. Here’s the screen I get after trying to install it:

And my phone’s specs:

Could you provide an updated version, please?

I figured that the current Alpha (2.7.269-alpha-2019-01-15) is identical so I QA’ed it anyway. Everything that @Sharvaniharan, @dchen and I discussed yesterday is working now. Minor detail in the “Translate title descriptions“ item copy in “App editor tasks“, which should say:

“Locked until you’ve edited 2 title descriptions“ instead of “Locked until you’ve made 2 verified in-app edits“

In regards to this @cmadeo:

You unlock title desc, you make a valid edit, you see the translation unlocked message and hit 'Get started' and go to translation edit screen, should back button land you in title desc screen or edit feed screen?

My guess is back to the edit feed screen as Title Description Translation is not nested within Title Description

I think @Sharvaniharan referred to the phone’s system back button (the one at the bottom of the screen) in this case. Or at least that’s how I understood it in the call. Then it makes more sense to take users back to where they came from (the Title description task list).

If she was referring to the app’s back button at the top left, then I agree with your suggestion @cmadeo to take users back to “App editor tasks“. Could you clarify @Sharvaniharan? Thanks.

CC: @Charlotte

Change 484548 had a related patch set uploaded (by Sharvaniharan; owner: Sharvaniharan):
[apps/android/wikipedia@master] Updated lock message for translate descriptions

https://gerrit.wikimedia.org/r/484548

Change 484548 merged by jenkins-bot:
[apps/android/wikipedia@master] Updated lock message for translate descriptions

https://gerrit.wikimedia.org/r/484548

Change 484606 had a related patch set uploaded (by Sharvaniharan; owner: Sharvaniharan):
[apps/android/wikipedia@master] Back button fix

https://gerrit.wikimedia.org/r/484606

Change 484606 merged by jenkins-bot:
[apps/android/wikipedia@master] Back button fix

https://gerrit.wikimedia.org/r/484606

Change 484758 had a related patch set uploaded (by Sharvaniharan; owner: Sharvaniharan):
[apps/android/wikipedia@master] [Do not merge] Dialog logic updated

https://gerrit.wikimedia.org/r/484758

Change 484770 had a related patch set uploaded (by Sharvaniharan; owner: Sharvaniharan):
[apps/android/wikipedia@master] Removes unused preference key

https://gerrit.wikimedia.org/r/484770

Change 484758 merged by jenkins-bot:
[apps/android/wikipedia@master] Dialog logic updated

https://gerrit.wikimedia.org/r/484758

Change 484770 abandoned by Sharvaniharan:
Removes unused preference key

https://gerrit.wikimedia.org/r/484770