@SChekfa-WMF Please take a look at these builds and let me know if they feel okay for user testing. This will be on the Experimental app in TestFlight, not the Staging app.

Non-donor build: Experimental 7.8.3 (278)

This build forces personalized data to match Figma, and forces a non-contributor state for the donate slide.

Donor build: Experimental 7.8.3 (279)

This build forces personalized data to match Figma, and forces a contributor state for the donate slide.

Test steps for users (this applies to both builds):

1. Device language in iOS Settings: English when targeting EN users, Japanese when targeting Japanese users.
2. Fresh install app for the first time. In app onboarding on the primary app language picker screen, have EN users choose English, and Japanese users choose Japanese.
3. If testing logged-in users, have user go to Profile from the Explore feed and log in. Have them use an account from https://docs.google.com/spreadsheets/d/1doKKVEWtsvrPoxZ32e9hREHD7sKb61h_s1vLjmgpJk0/edit?gid=0#gid=0.
4. Visit an article, Year in Review feature announcement should appear. Go through Year in Review flow. Ensure you see what is expected, and that mock data looks correct.

Things that are not yet ready

1. I don't have Japanese translations in yet - the spreadsheet had missing translations. Maybe I don't have the proper access? I will make followup builds once those are in.
2. We are still working on design review feedback of https://phabricator.wikimedia.org/T402696
3. We are also still working on design review feedback of https://phabricator.wikimedia.org/T402693

Let me know if any of those items in #2 and #3 are nice-to-have for the user testing build (we will still work on them for the production release).

@Tsevener earlier on Slack you mentioned that if they have the app language set to English or Japanese, they don't need to go into device settings — is that still true? i noticed the test steps include device language here. I tried just doing it in onboarding like mentioned before and it didn't reformulate to Japanese. (no worries if they need to go into device settings, but i'm going to need to specify that in user testing so just want to confirm with you)

also, for logged-in users, how would they access their account? did you choose a specific account in the file to link the fake data to? tbh i'm going through them and it keeps telling me the passwords are wrong!

re: #2 and #3, #2 is definitely not necessary for user testing but #3 would be ideal to include since those improvements would present a more polished final screen reflective of the v3 design to more accurately compare v2 against.

also @Tsevener, will users need to be added to Testflight in order to access the Experimental build? I'm wondering if we'll need them to provide their emails in advance — I recall I provided my email address when I first started using the app but maybe there's a work-around for situations like these?

@SChekfa-WMF

https://wikimedia.slack.com/archives/C4DDMJ9CH/p1757686479038339

We need their device language to be set to Japanese in order for them to see the YiR copy in Japanese. It is a bit confusing but this might help:

1. Device language -> Controls copy translations
2. App language -> Controls whether they see the English microsite slides or not, also controls which Wiki we pull edit counts from.

Hopefully that clears things up. I think ideally for Japanese users they should have both their device language and primary app language set to Japanese.

also, for logged-in users, how would they access their account?

If a user is already logged in (say, they already have the App Store Wikipedia installed on device, and we have them update to this newer user testing build), no special login needs to happen. They are good. I am overriding the actual edit counts that get saved in the year in review report. Logging in for these testers is purely so that the other areas of the app are in a correct logged-in state (like Profile, Settings, etc).

If a user is NOT logged in, they will need to log in with some kind of test account, maybe from that spreadsheet. I guess worst-case-scenario we can just create a new test account. But I am not targeting any particular account in the app. We just want them to log in with any account so that the app reflects a logged-in state. Then I am overwriting their editing counts with the fake user testing data.

I have another question - are you going to test the login via the feature announcement? Where they tap "Get Started" and they see a log in prompt - would you rather have them log in through that instead of through Profile?

will users need to be added to Testflight in order to access the Experimental build? I'm wondering if we'll need them to provide their emails in advance — I recall I provided my email address when I first started using the app but maybe there's a work-around for situations like these?

(Just documenting from Slack records)

Let's try to have them test against the main Wikipedia TestFlight app. This way they won't be required to uninstall the app and potentially lose their Year in Review data for the production release later this year.

@Tsevener thanks for breaking that down!

you mentioned: "If a user is already logged in (say, they already have the App Store Wikipedia installed on device, and we have them update to this newer user testing build), no special login needs to happen. They are good. I am overriding the actual edit counts that get saved in the year in review report. Logging in for these testers is purely so that the other areas of the app are in a correct logged-in state (like Profile, Settings, etc)."

just to confirm, we will want the users to all log in using the same account (i'm including in the setup the username/password you found). they will all be getting the same instructions, and i'd prefer to avoid some staying logged into their account and writing special instructions for those who are in that scenario and those who don't have accounts, as it will overcomplicate and likely confuse users. I will just include a note at the beginning asking them to log out of their account if they are currently logged in. does that work for you?

also, re: your Q "I have another question - are you going to test the login via the feature announcement? Where they tap "Get Started" and they see a log in prompt - would you rather have them log in through that instead of through Profile?" yes, I want them to log in through the "Get started" in the flow because one of the research questions Jaz wants answered is around the clarity of that flow. does that work for you?

also, for EN users, should they also go into device settings and ensure they are set to US as Region and English as primary language? @Tsevener

@SChekfa-WMF

just to confirm, we will want the users to all log in using the same account (i'm including in the setup the username/password you found). they will all be getting the same instructions, and i'd prefer to avoid some staying logged into their account and writing special instructions for those who are in that scenario and those who don't have accounts, as it will overcomplicate and likely confuse users. I will just include a note at the beginning asking them to log out of their account if they are currently logged in. does that work for you?

That makes sense. The only maybe annoying thing is our app presents confirmation modals when you try to log out if you have saved articles + syncing on. So just a heads up there, be sure to have instructions that accounts for that when asking them to log out. I wouldn't want them to accidentally delete their saved articles somehow.

Here's what I see when I log out while having saved articles + syncing. 3 modals 😩:

yes, I want them to log in through the "Get started" in the flow because one of the research questions Jaz wants answered is around the clarity of that flow. does that work for you?

Will do, I have a bug to fix when going through the "Get started" > "Login" flow (I think editing slides may not show), so I will work on that and produce another build with that fixed.

also, for EN users, should they also go into device settings and ensure they are set to US as Region and English as primary language?

For EN users, we just need them to check devices settings and ensure they have English as a primary language. Then have them check app settings and ensure EN is set as primary app language.

Same goes for Japanese users. Check devices settings and ensure they have Japanese as a primary language. Then have them check app settings and ensure Japanese is set as primary app language.

We are more flexible on device region now. Any device region set up in the config should show Year in Review. The config (based on last year's) targets US, GB, IE, AU, CA, and JP. I'm fuzzy on UK and if that is a recognized device region in the Apple API, I will look at get back to you. But assuming these users already have one of those regions set in their device settings, the feature will show for them.

@Tsevener re your screenshots, why is it asking them to log in to sync after they've logged out? shouldn't it have synced already? I guess I can include those instructions over email outside of Userlytics specifically for people who say they have an account, as we will be screening for that so can avoid confusing people who don't fall in that bucket. are you saying the instructions would be logout > keep articles on device > then close that log-in dialog? all within the default Wikipedia app, not within the app they'll be testing in, correct?

so re: device settings, they don't need to touch region then? only language? and then within app settings, during onboarding at the language, they can select their primary language (so I don't have to instruct them to go into settings and do it from there)?

thank you!

@SChekfa-WMF

re your screenshots, why is it asking them to log in to sync after they've logged out? shouldn't it have synced already?

It's possible that it displayed after I logged in as iOS-Test-Eng. I had a saved article (because I chose to not delete my saved articles upon logout) with syncing turned off, so maybe it was asking me to turn syncing back on. I couldn't trigger it again though, so not sure what the business logic is on that modal, it's an old one. My advice would be to carefully go through your scripts yourself and lookout for any confirmation dialogs that pop up as you test and note them in the script.

are you saying the instructions would be logout > keep articles on device > then close that log-in dialog? all within the default Wikipedia app, not within the app they'll be testing in, correct?

They could log out on either app. Either log out on the default Wikipedia app before they update via TestFlight, or log out / log into other account after they update via TestFlight.

Alternatively, you could let them stay logged into their account after updating. I think technically the app would still work okay, we would still fake their YiR data with the same numbers.

re: device settings, they don't need to touch region then? only language?

Correct, they only need to touch device language.

and then within app settings, during onboarding at the language, they can select their primary language (so I don't have to instruct them to go into settings and do it from there)?

I just realized if we are allowing them to update to this build from the current App Store build (to preserve their YiR Data) then they likely would have already gone through app onboarding and they won't see it presented again. So you'll have to instruct them to update their primary app language in app Settings.

@SChekfa-WMF

Here are the user testing links:

Donor build:
https://testflight.apple.com/join/FKAe2N5y

Non-donor build:
https://testflight.apple.com/join/F58ahNaM

They have been updated with my latest PRs (bug fix with the Get Started > Login flow mentioned in https://phabricator.wikimedia.org/T402319#11191325, and applied design review feedback fixes for the highlight slide). Both should update the App Store app if you already have it installed.

@Tsevener thank you, I will review these links.

I'm thinking that for the sake of simplicity, we just ask them to delete their Wikipedia app so they always see the onboarding — I'm just nervous about having so many caveats in the testing protocol, and want to make it as simple as possible. also, this is the set-up that we've used historically, per Olga's protocol, so I'd like to reduce confusion and maintain clarity by going down that route. If I ask them to delete their app if they have one at the very beginning of the test, then they'd definitely see the onboarding once they download the app we'll be testing through via Testflight, correct?

also, another question — can anyone access the build through these links, or do they have to be added to a testing group before downloading?

also, i'm trying to review right now but the YIR feature is not loading for me. I am doing a fresh install, no prior Wikipedia app, but whereas it was working in Experimental yesterday this new build does not load the feature upon download. Do you know what might be going on? given that this is the build for users, they should not have to be going into developer settings, so I feel that this is a bug:

video:

https://drive.google.com/file/d/1XHCt1oAYy-n1xUfpmINxwQm6BaGwJp8d/view?usp=sharing

@Tsevener

Hi @SChekfa-WMF -

I think because you are an internal tester, you have access to more builds than the average person that comes in through those links. When I access the link through my personal device (which has an AppleID that is not set as an internal tester), I see this:

The builds you want to seek out are:

7.8.3 (5833) for Donor
7.8.3 (5834) for Non-Donor

Install those builds directly and it should work.

If I ask them to delete their app if they have one at the very beginning of the test, then they'd definitely see the onboarding once they download the app we'll be testing through via Testflight, correct?

Correct

can anyone access the build through these links, or do they have to be added to a testing group before downloading?

Anyone can access the build through these links.

ah ok TY it's working now.

some notes:

donor build
logged in EN users are not seeing the Reading Patterns screen. please include this after your Top articles and before Categories.

logged in non-EN users are not seeing the Reading Patterns screen. please include this after your Collective articles viewed on apps Non-EN and before Categories.

the same is true for the non-donor build:
logged in EN users are not seeing the Reading Patterns screen. please include this after your Top articles and before Categories.

logged in non-EN users are not seeing the Reading Patterns screen. please include this after your Collective articles viewed on apps Non-EN and before Categories.

@Tsevener

@Tsevener i'm also realizing that we will need to swap out the editing screens for the non-editing slides for the non-donor build, since editing is one way to unlock icons and it's simpler to just assume everyone hasn't done anything to earn the icon, be it editing or donating.

can we swap out this editing screen and this editing screen for their respective non-editing screens for the non-donor build?

for EN these are the non-editing screens (1, 2)
for non-EN these are the non-editing screens (1, 2)

this would also mean that the # of edits would be redacted from the highlights infobox.

TY!! sorry I didn't catch this before.

@SChekfa-WMF
Latest User Testing Builds:

Wikipedia (white icon)

Donor: 7.8.3 (5836)
Non-Donor: 7.8.3 (5837)

This has Japanese translations and fixes from your comments above. It does not have your latest feedback in https://phabricator.wikimedia.org/T402693#11194528, https://phabricator.wikimedia.org/T402693#11194674, and https://phabricator.wikimedia.org/T404750#11194735.

@SChekfa-WMF

Experiemental app was approved for external beta testing. Here are the user testing links for the Experimental app:

Here are the user testing links:

Donor build:
https://testflight.apple.com/join/ddwgYUty
Latest build will be 7.8.3 (282)

Non-donor build:
https://testflight.apple.com/join/Kbr36nRX
Latest build will be 7.8.3 (283)

Same notes apply as earlier, it has Japanese translations, but does not have your latest feedback in https://phabricator.wikimedia.org/T402693#11194528, https://phabricator.wikimedia.org/T402693#11194674, and https://phabricator.wikimedia.org/T404750#11194735. Please let me know if any of those are must-have for user testing, thanks!

EDIT: Some items from your feedback are addressed. You should see "Times edited" and 50%. The rest hasn't been addressed yet.

hi @Tsevener, i'm reviewing now but just want to flag that you don't need to include those last highlights improvements in the user testing build, it's fine to just address for production.

@SChekfa-WMF

Latest builds with fixes from https://phabricator.wikimedia.org/T404750#11194735 plus a Japanese translation fix.

Donor build:
https://testflight.apple.com/join/ddwgYUty
Latest build will be 7.8.3 (286)

Non-donor build:
https://testflight.apple.com/join/Kbr36nRX
Latest build will be 7.8.3 (287)

these look great, TY Toni! : ) @Tsevener