@alexhollender, @jwang - I think we might have some data on this from the beginning of the project, although the updated numbers would also definitely be helpful. @jwang - when looking into this, would it be possible to split the data into edit buckets? My intuition says that the number might be higher for more veteran editors.

Hi @alexhollender , as we discussed on slack, data instrumentation is only enabled on 14 pilot wikis with 20% sample rate. Wikidata and commons are not one of pilot wikis, therefore we don’t have data for Wikidata and Commons.

Following data is collected from the 14 pilot wikis between January 1st, 2021 and July 31st, 2021 and categorized by the wiki family. It shows on wikis in wikipedia, wikivoyage and wikiversity, watchlist is most often clicked link (>40%) , while contributions is at the second place. On wikis in the wiktionary and incubator family, Contributions is the most clicked link.

Have analyzed the impact of user menu deployment on pilot wikis by following dimensions.

• What is the frequency of clicks to these links?
• What is the pattern from before/after implementing the user menu?
• Does edit count affect this in any way?
• What is the pattern if split the data into edit buckets?

Conclusions: The trends are in line with our expectations.
Analysis report & code: https://nbviewer.org/github/jenniferwang-wmf/WEB_user_menu_2021/blob/master/User_menu_logged-in_user_before_after_report.ipynb

Re-analyzed by skin version, especially focused on new skin version. We found after deployed new user menu, the numbers of clicks on user links by users with new skin decreased on all types of links. The changes varied between -48% ~ -80%.

We cannot confirm whether the issue T300901 impacted our deployment analysis because the historical data of skin preference changes has been wiped.

Report:
http://nbviewer.org/github/jenniferwang-wmf/WEB_user_menu_2021/blob/master/User_menu_logged-in_user_before_after_report-by_skin_version.ipynb

In below report, we reviewed the total number of clicks per init events on each available user tools by each pilot wikis. The initiation events are recorded in DesktopWebClickTracking schema. The init event is fired for everyone who’s eligible for click tracking. The rate of clicks per init events is the click through rate of the users who clicked the links out of users who are eligible to click. We found after the deployment the click through rate decreased on all types of links which have moderate traffic. The changes varied between -47% ~-84%. It's consistent with the trend we see in the number of the clicks mentioned in report2.

Report: http://nbviewer.org/github/jenniferwang-wmf/WEB_user_menu_2021/blob/master/3_User_menu_logged-in_user_before_after_report-new_skin_version.ipynb