@atgo Per the earlier discussion at T185584, I have clarified the scope of this task as assessing the impact of the design change, rather than assessing the impact of the video campaign. (Question #3 in T185584 still covers the latter regarding search.)

Hi @mpopov @chelsyx when do you think you'll be able to look into this and T191354? Ideally I'd like to have some numbers before the QCi in 2 weeks.

In T191132#4155475, @atgo wrote:

Hi @mpopov @chelsyx when do you think you'll be able to look into this and T191354? Ideally I'd like to have some numbers before the QCi in 2 weeks.

Checking again

Hi @mpopov @chelsyx when do you think you'll be able to look into this and T191354? Ideally I'd like to have some numbers before the QCi in 2 weeks.

Hi @atgo , I should be able to look at it this week

Thanks @chelsyx

I'll be OOO starting on Thursday and @ZMcCune will be taking over on tracking this for me.

Copy from T191354#4176890

We don't see any spike or dip in the total number of search, zero result rate and clickthrough rate, when the Hindi mobile main page change (March 26th), or when the video campaign launch (April 2nd):

In sum, we don't see any impact on search from mobile main page change or video campaign on Hindi Wikipedia.

@atgo Please let me know if you have any question.

Thanks @chelsyx . Apologies for the very slow response time on this, I've been OOO on sabbatical and just back now.

There are a couple of things I'd like to understand more here.

1. There is a steady drop in app searches across this time. What's that all about?
2. App searches seem incorrectly high as compared to desktop or mobile web. In your first screenshot, they look like 2-4x more searches from the app, but the web's traffic overall looks like about 100x that of the app, so I'm finding this hard to believe. Can you clarify what's going on here?

In T191132#4423594, @atgo wrote:

Thanks @chelsyx . Apologies for the very slow response time on this, I've been OOO on sabbatical and just back now.

There are a couple of things I'd like to understand more here.

[...]

App searches seem incorrectly high as compared to desktop or mobile web. In your first screenshot, they look like 2-4x more searches from the app, but the web's traffic overall looks like about 100x that of the app, so I'm finding this hard to believe. Can you clarify what's going on here?

Agree that it would be good to know the answer to this question. (Also, are the data source and queries used for this analysis documented somewhere? Does it have to do with the dataset mentioned at https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Cirrus ?)

Another thing: Since this task was about changes to the mobile web version specifically, it could have been worth separating mobile web and desktop searches in F17605560. Considering the outcome here, it may not yield more insight in this specific case, but it may be worth keeping in mind for T199726: Understand if ru.wikipedia main page changes had impact on mobile users.