→ Related to T253238

Thx @SNowick_WMF

Any additional info on metrics available would be great to have.

I’m not aware of prior reports / studies, but maybe @Dbrant knows more?

Is there a time frame or app version we specifically want to look at (recent, less recent?)

Recent is fine!

In T253240#6153914, @SNowick_WMF wrote:

I will need to find where and what data on these actions is stored, if there are prior reports/studies on navigation to use as baseline, etc. Any additional info on metrics available would be great to have.

We last really looked into this about 2 years ago when we made the last round of navigation improvements. Things have changed so much since then that it's not really comparable. The instrumentation should - fortunately - exist, however, since we were able to get numbers before.

Is there a time frame or app version we specifically want to look at (recent, less recent?)

Let's look since 25 November 2019, when SE was put in the menu for logged-in users.

@Charlotte @SNowick_WMF

Just wanted to flag this task as we just released a version to Wikipedia Beta that includes a drastical change to the navigation.

It would be a pity to not be able to observe and comprehend the impact of the changes on usage of nav items.

Before	vs	After

I am adding my notes here on where analytics may have data on interaction with Navigation. For the other items requested originally, we don't have a 'click tracking' data set to measure each interaction across the app, there is some info on Main tab navigation in Schema: MobileWikiAppSessions (see T186682 ) and for Search we have Schema: MobileWikiAppSearch but those are very basic metrics. Unless we plan ahead, knowing we want to measure engagement, and put in analytics calls to collect interaction data, we don't get stats.

For the Nav change:

Schema: MobileWikiAppFeedConfigure
--It looks like Customize the Feed is now one layer deeper under Settings? We can measure if use increases or decreases based on that change.

Schema: MobileWikiAppNavMenu

--Not sure if the metrics we were getting from the left Nav bar will be carried over to the new Nav bar, also not sure this is the same Nav Menu or it's metrics for something else, possibly the More menu that pops up, not the main navigation items at the left/bottom.

Schema: MobileWikiAppAppearanceSettings

• This is accessible through Navigation under Settings but also accessible from reading an article so may not be a useful metric since we don't track where the click comes from

I will collect and report on what data we do and do not have coming soon but in general if changes are made and we know we want to measure the result loop me in before the change is made and we can set up analytics to track engagement.

I have queried the MobileWikiAppNavMenu data table and below is representation of events that are logged, it looks like this is actually the Navigation Menu that's available at the bottom of the app, rather than the navigation feature that @schoenbaechler has made changes to. The counts also look very low, even with a sampling rate of 1/100 (as noted in T104623. Will continue to look for data points associated with the menu requested. I've set up a Superset Dashboard to collect data from tables listed above that may have possible data relevant to this request. https://superset.wikimedia.org/superset/dashboard/androidnavdata/

Thanks @SNowick_WMF.

it looks like this is actually the Navigation Menu that's available at the bottom of the app, rather than the navigation feature that @schoenbaechler has made changes to.

Hmm, not sure I understand. This change is about the bottom navigation as well, see T253238. More image material to illustrate what has changed:

Before (“More” menu on Explore)

After (“More” menu in main tab bar)

More specific changes:

• Add “More” item to bottom navigation.
• Use bottom navigation drawer instead of side navigation drawer.
• New bottom navigation drawer is accessed via “More” item.
• Restyle header with username to be displayed on one line
• Remove “Customize the feed” option from navigation drawer
• Add “Explore feed” to “Settings” (Copy: “Customize the Explore feed”)
• Remove shadows from top nav bar
• Swipe anywhere in feed no longer opens this menu
• How do measure success?
• See subtask: T253240
• Compare usage of drawer before and after
• Compare usage of items in drawer before and after (access of/taps on individual items)
• Especially interesting: “Support Wikipedia”

Happy to jump on a call to discuss it if it’s unclear @SNowick_WMF.

Thanks again

Is any of the data shown on the Android Navigation dashboard relevant to these items listed? We can schedule a call but so far I am not seeing where we have data to track any event outside of the Feed, Login, Main Page, More, Nearby, Random, ReadingLists, Recent, SavedPages & Today events.

Search
I'm adding to this as I pull data so it's a bit ad hoc but this will give you visibility into what tracking we do have.
From table MobileWikiAppSearch we have tracking from search starts from the following locations, usage counts shown on chart: App shortcut, Floating search bar in the feed, Main article toolbar, Process-text intent, Share intent, Voice search query and Widget.

In T253240#6302681, @SNowick_WMF wrote:

Is any of the data shown on the Android Navigation dashboard relevant to these items listed? We can schedule a call but so far I am not seeing where we have data to track any event outside of the Feed, Login, Main Page, More, Nearby, Random, ReadingLists, Recent, SavedPages & Today events.

What do the numbers in those charts represent? Users? Literal number of times a menu item was selected? Either way, the values seem several orders of magnitude too low - I can't believe that for the week of Monday July 6th, the Randomiser was only selected by 11 people/selected 11 times. Unless they are thousands?

It would be really useful to have a bit of context and some recommendations!

Figured out this schema was superannuated, so we're resolving this ticket - we've got just about all the navigation info we can get without painful reconstructive efforts.