Page MenuHomePhabricator

research: Understand how current and new users of Growth features would be affected by switching to Vue
Closed, ResolvedPublic

Description

There are some features that we can safely migrate to Vue, for example, the Mentor Dashboard as less widely available feature that is targetting a specific subset of users.

However for the following features:

  • help panel
  • suggested edits
  • impact (which will be rewritten as part of positive reinforcement project)
  • ask question

We will want to understand how this affects our existing userbase. Given that Vue features will use Vue 3, which is not supported on IE11 (https://www.mediawiki.org/wiki/Vue.js#IE_11), we need to look at our existing userbase and understand how they would be impacted by switching e.g. suggested edits to Vue.

Event Timeline

kostajh triaged this task as Medium priority.Feb 7 2022, 6:56 PM

Do you all have traffic numbers for the features under consideration here? Might be helpful in trying to determine the best way forward.

According to the data here, IE users count for ~0.5% of overall traffic and ~1.2% of desktop traffic. Interestingly the IE numbers dropped off significantly in early 2020 – I wonder if that corresponds with people leaving schools/offices (where they used old PCs) to work at home during the pandemic.

Do you all have traffic numbers for the features under consideration here? Might be helpful in trying to determine the best way forward.

According to the data here, IE users count for ~0.5% of overall traffic and ~1.2% of desktop traffic. Interestingly the IE numbers dropped off significantly in early 2020 – I wonder if that corresponds with people leaving schools/offices (where they used old PCs) to work at home during the pandemic.

We don't have traffic numbers on hand yet. We need to look specifically at logged-in users, as that is who we are focusing on with the relevant features.

Hello! Here is a related task around IE11 user behavior that I've been exploring with the Analytics team: T303325: IE11 User and Traffic Analysis

It would be great to move off of IE11 in a MW release following Microsoft's official deprecation. We can continue to watch the traffic to confirm downward trends after June 15.

Sgs changed the task status from Open to In Progress.May 20 2022, 11:01 AM
kostajh raised the priority of this task from Medium to High.May 31 2022, 7:59 PM

Updating the priority of this, as we need the analysis in order to be able to move forward on questions like deciding whether to use e.g. Pinia as a store, or if we can migrate features like PostEditDialog to Vue/Codex. @Sgs perhaps we could try to resolve this task by the end of next week?

Updating the priority of this, as we need the analysis in order to be able to move forward on questions like deciding whether to use e.g. Pinia as a store, or if we can migrate features like PostEditDialog to
Vue/Codex.

We spoke with @mewoph about the use of Pinia vs Vuex and we had the impression we didn't have enough objective arguments to move from current implemented Vuex stores to Pinia.

@Sgs perhaps we could try to resolve this task by the end of next week?

Yes.

Sharing some early results. The data gathering can still be refined but we can start to see some useful information from the "homepage visit event" queries across wiki projects and countries. The Jupyter notebook used can be found here.

Analysis description

The data collection has been made for ~200 wiki projects for the month of May 2022. We are counting the number of users visiting the homepage with GE features enabled and using IE11 and calculating a rate from the total number of visitors.

Analysis results

  • The collected data shows kowiki (Korean wiki) as the highest rate of IE11 users with a 1.735358 % ( from 922 users in May, 16 were using IE11). The next project is jawiki ( Japanese wiki ) with a rate of 0.405885 % and the rate dilutes quickly after the next projects getting close to 0 .
  • The collected data shows Sudan, South Korea, Syria and Philippines as the countries with highest rates of IE11 users with rates around 1-1.5 %. The next countries are Myanmar, Japan and Uganda with rates around 0.4 %

Full results:


Analysis improvements

  • Expand dates of analysis, run multiple times
  • Add other deprecation candidate browser families, see thread

Conclusions

Given the rates we're seeing I would suggest to move forward with deprecating IE11 as a supported browser for GrowthExperiments features. I would suggest to communicate this to the most affected projects/countries ( Help would be appreciated on knowing what would be the best way to do it @Trizek ).

The effort to keep giving support to this users on the long term would conflict directly with our plans to migrate things to Vue 3. An interim solution could be to use a build step to add polyfills for IE11 code but that has it's own inconvenient and seems not worth for this case ( it could make sense for other goals like using TypeScript).

Given we (as the WMF) are planning to use "native" Vue SSR (server side rendering, see T286966: Prototype a Vue SSR implementation using V8js ) we will always be able to give some degree of feature support to IE11 users by creating "meaningful content first renders" and relying on our existing "no-javascript" views.

What do you think about dropping IE11 support from next fiscal year on GrowthExperiments features? @MMiller_WMF @kostajh

What would break? If users can't click on a button when they use IE11, then it is a problem. But if nothing changes expect a bit of the layout, then who cares? :)

Also, we have an official compatibility support at an organization level. It says that we still support IE11, except for JS. Will it change in regards of Microsoft's official deprecation? To make it short, I don't think it is our call, but an upper level one: what is the new official position of the WMF regarding IE11 support?

What would break? If users can't click on a button when they use IE11, then it is a problem. But if nothing changes expect a bit of the layout, then who cares? :)

IE11 users would not be able to interact with the interface for the Impact module at all. They would only have whatever content we render along with the page load.

Also, we have an official compatibility support at an organization level. It says that we still support IE11, except for JS. Will it change in regards of Microsoft's official deprecation? To make it short, I don't think it is our call, but an upper level one: what is the new official position of the WMF regarding IE11 support?

Per the official decision, we don't really need to do this task because the guidance says we don't need to accommodate IE11 users with new UIs written in JavaScript.

But since the numbers cited in https://www.mediawiki.org/wiki/Compatibility/IE11 are looking at overall page traffic, and not specifically at our user base, we thought it would be wise to double-check how our actual users would be impacted by this policy.

IE11 users would not be able to interact with the interface for the Impact module at all. They would only have whatever content we render along with the page load.

I believe we can continue to support a "functional" Impact module for IE11 users because most of the features are "read only". The article titles and page visits numbers and even some graphics (those constructed with compatible CSS as opposed of JS) could work for IE11, Safari 9, 10, etc. As per the "filtering" options placed in "dropdown" or similar elements ( see T220139: New Impact Module: Filter/Sort options to customize the top edited articles of the impact module ), those would require more work (ie: handle form POST action data) but could even be functional with a native select element. Just to surface that there are several alternatives to continue giving support to old browsers that are compatible with our plans to modernize client JS code.

kostajh added subscribers: Mayakp.wiki, Trizek.

Sharing some early results. The data gathering can still be refined but we can start to see some useful information from the "homepage visit event" queries across wiki projects and countries. The Jupyter notebook used can be found here.

Analysis description

The data collection has been made for ~200 wiki projects for the month of May 2022. We are counting the number of users visiting the homepage with GE features enabled and using IE11 and calculating a rate from the total number of visitors.

Analysis results

  • The collected data shows kowiki (Korean wiki) as the highest rate of IE11 users with a 1.735358 % ( from 922 users in May, 16 were using IE11). The next project is jawiki ( Japanese wiki ) with a rate of 0.405885 % and the rate dilutes quickly after the next projects getting close to 0 .
  • The collected data shows Sudan, South Korea, Syria and Philippines as the countries with highest rates of IE11 users with rates around 1-1.5 %. The next countries are Myanmar, Japan and Uganda with rates around 0.4 %

Full results:


Analysis improvements

  • Expand dates of analysis, run multiple times
  • Add other deprecation candidate browser families, see thread

Conclusions

Given the rates we're seeing I would suggest to move forward with deprecating IE11 as a supported browser for GrowthExperiments features. I would suggest to communicate this to the most affected projects/countries ( Help would be appreciated on knowing what would be the best way to do it @Trizek ).

The effort to keep giving support to this users on the long term would conflict directly with our plans to migrate things to Vue 3. An interim solution could be to use a build step to add polyfills for IE11 code but that has it's own inconvenient and seems not worth for this case ( it could make sense for other goals like using TypeScript).

Given we (as the WMF) are planning to use "native" Vue SSR (server side rendering, see T286966: Prototype a Vue SSR implementation using V8js ) we will always be able to give some degree of feature support to IE11 users by creating "meaningful content first renders" and relying on our existing "no-javascript" views.

Nice work, @Sgs! Thank you for doing this analysis.

I am a little surprised to not see a stronger correlation between the data that the notebook generated and the numbers cited in T303325#7812013 (cc @Mayakp.wiki). On the other hand, we are looking at a narrower subset of traffic, so it is not overly concerning.

What do you think about dropping IE11 support from next fiscal year on GrowthExperiments features? @MMiller_WMF @kostajh

The analysis here makes me feel confident in dropping IE11 support in GrowthExperiments features and moving forward with Vue migration of our code. @KStoller-WMF, could you please have a look and let us know your thoughts as well? Then we could mark this task as resolved.

The best thing we could do is to invite people to switch to a different browser, if they can. But again, it is a global move to make.

Thanks for this analysis and summary, @Sgs !

The analysis here makes me feel confident in dropping IE11 support in GrowthExperiments features and moving forward with Vue migration of our code. @KStoller-WMF, could you please have a look and let us know your thoughts as well? Then we could mark this task as resolved.

This summary, combined with Microsoft's official deprecation coming up in in a few days (June 15, 2022) makes this seem like a fairly safe choice. My assumption is that Microsoft will already be showing a very clear "update browser" call to action for users still on IE11.

I don't have enough context to know if this is a decision we can make as a team or if we need a higher level sign-off on dropping support for IE 11 as a team, so let me know if there is anything further I should be following up on.

I don't have enough context to know if this is a decision we can make as a team or if we need a higher level sign-off on dropping support for IE 11 as a team, so let me know if there is anything further I should be following up on.

I don't think any further approval is needed. Sharing the analysis with other product teams might be worthwhile.

kostajh renamed this task from research: Understand how our current and new users would be affected by switching to Vue to research: Understand how current and new users of Growth features would be affected by switching to Vue.Jun 29 2022, 11:00 AM