Following discussion with @Sharvaniharan and @Dbrant the plan for this ABC test is to use the MobileWikiAppNavMenu schema to track views and clicks for each variant Notification icon. Since we can't target UI by geolocation/wiki specifically, we will filter the results by wiki and geocoded_data based on the feedback criteria groups at the tail end.

For action field we added a show event, the menuItem field corresponding to the event will indicate which icon was shown or selected: (A) inbox, (B) bell and (C) more.

I’m worried that the pool of users in all regions except English is too small to get to any statistical significance @SNowick_WMF — what do you think?

Re: designs and positioning - this looks good to me. @Dbrant is there a way to check out the designs in an APK’s?

Thanks!

@schoenbaechler Just a few clarifications around the third variant:

• This mock shows the Main screen, but we currently don't have an overflow menu on the main screen. Did you mean to put the Dot indicator over the "More" menu, which is where the Notification item actually lives?
• When reading an article, we do have an overflow menu in the toolbar with several items in it. Are we adding "Notifications" as another item to the menu? Which position in the list would it be?

Thx @Dbrant, answers below:

This mock shows the Main screen, but we currently don't have an overflow menu on the main screen. Did you mean to put the Dot indicator over the "More" menu, which is where the Notification item actually lives?

Yeah, let’s test it like this.

When reading an article, we do have an overflow menu in the toolbar with several items in it. Are we adding "Notifications" as another item to the menu?

Yes!

Which position in the list would it be?

Below the 'Explore' menu item.

In T288248#7281021, @schoenbaechler wrote:

I’m worried that the pool of users in all regions except English is too small to get to any statistical significance @SNowick_WMF — what do you think?

The user segments are small but since this experiment is going to reach all users we will be able to pull results from bigger user segments for comparison.

@schoenbaechler This ABC test branch is now ready for testing, even while we continue to code-review it:
https://github.com/wikimedia/apps-android-wikipedia/actions/runs/1136719330

To change between different test groups (0, 1, or 2) go to Developer preferences, and scroll all the way down to the "A/B testing" section.

@Dbrant, great & thx for providing the link. I checked it out.

01) Please remove the notifications menu item from 'More' in both version 0 and 1:

The idea is to remove it from the 'More' menu item completely if one of the variants is succesful

02) It seems like you’re using a different color for the notification circle/background (see images below)? Please make sure to use color_group_44 from the theme guidelines.

03) could you slightly increase the size of the notifications circle and number, everywhere it’s used to make it more legible? I use a circle width/height of 22px and Roboto Mono Bold 11px as the font size. If Roboto Mono Bold is not feasible (due to the bug you once mentioned) use Robot Bold. Check out the designs to get an idea

https://www.figma.com/file/NahRwTAJhSV2B08g7TaDvs/Android-Notifications?node-id=284%3A11796

@schoenbaechler The mock you posted now has the dot placed at the top of the bell icon, but the previous mocks had it at the bottom. What placement do we want? Is it the same for all the other locations?

@Dbrant — dot placed at the top of the bell icon is the preferred way as the bell icon is more recognizable this way

@schoenbaechler Done; all points addressed:
https://github.com/wikimedia/apps-android-wikipedia/actions/runs/1139529032

@Dbrant just checked all three variants, looks good to me now! 🎉

Looks good to me. FYI → Instructions to try it out are here: T288248#7286309

Note: These changes move Notification Preference settings out of the app and onto the device, so tracking second-level settings changes will no longer be possible.
We will instead need to track user setting change for all_notifications option in the new UI.

Not related to ABC test but I have a question about non-Notifications setting items that are on the Settings 3 on the Figma diagram. There are options to change settings for "Syncing Reading Lists" and "Downloading articles", these settings are also accessible from the Settings Menu. Will need to confirm that we can track when they are accessed from the new Nav Bar UI but this may not be the case.

It looks like we may not be currently tracking these settings but I was planning to add them to the new MEP android_setting_action and android_users_state schemas. Can we look into these and plan to track in next version?

In T288248#7329819, @SNowick_WMF wrote:

Not related to ABC test but I have a question about non-Notifications setting items that are on the Settings 3 on the Figma diagram. There are options to change settings for "Syncing Reading Lists" and "Downloading articles", these settings are also accessible from the Settings Menu. Will need to confirm that we can track when they are accessed from the new Nav Bar UI but this may not be the case.

To clarify, "Syncing reading lists" and "Downloading articles" are in fact notifications. When reading lists are being synced, or articles being downloaded, we show a "notification" with a progress bar that indicates the progress of the syncing or downloading. As with any other type of notifications, these can be suppressed by the user at the system level, which is what you see in the screen you mentioned. The important thing is that disabling those "notifications" does not actually mean that syncing of reading lists will be disabled, or that articles won't be downloaded.

Moving this off from QA, since the instructions for QA are specified in T287113.

Hey @Dbrant can we disable the ABC test for notification placement as soon as possible? I’m running usability tests for talk pages with the production version this week and the notification icon should be at top right for it. Thanks!