Page MenuHomePhabricator

[SPIKE] Determine project testing environments
Open, Needs TriagePublic

Description

This task involves the work of defining the environments (read: device(s), browser(s), operating system(s), etc.) we will use in the regression testing and patch verification we will do throughout the course of this project.

Requirements

References

Dashboard that shows the number of edits from countries identified within the Sub-Saharan Africa region by browser, os and device: https://superset.wikimedia.org/r/2157

Recommendation

Open questions

  • 1. For Editing Engineering: If we assume that the operating systems (Android) and browsers (Chrome) are consistent across the range of devices people are using, what deviations might we not be able to detect were we to omit testing on a device like the Xiaomi Redmi 10. See Editing Team/Edit Check for more context.

Done

  • The Recommendation section is populated with the proposal described in the Requirements section
  • File the requests needed for Editing Team quality test engineers to gain access to the devices we will need to test with throughout the course of this project

Event Timeline

ppelberg edited projects, added Editing-team (Kanban Board); removed Editing-team.
ppelberg updated the task description. (Show Details)
ppelberg added a project: Editing QA.
ppelberg moved this task from Incoming to QA on the Editing-team (Kanban Board) board.

Per what @Ryasmeen and I discussed offline today, we're going to do the following to ensure the new features we'll be introducing as part of this project will work as we expect on the browsers, devices, and operating system the people who we are centering in this work are most likely to be using:

Now: once T325331 is resolved, we're going to identify the countries within SSA where the majority of people who are contributing to Wikipedia are living so that we can then look at the devices they are using. With a list of devices "in hand" we'll compare it with the WMF's existing device inventory to determine what new devices we need to purchase so that we can be sure we're testing the new features we're developing on the devices the people we are centering in this work are likely to be using on them.

Later: ahead of each significant deployment (e.g. introduction of a new feature), we're going to look at the operating systems and browsers the majority of people from within SSA who contribute to said deployment wikis are using and set up our testing environments to match them.

Now: once T325331 is resolved, we're going to identify the countries within SSA where the majority of people who are contributing to Wikipedia are living so that we can then look at the devices they are using. With a list of devices "in hand" we'll compare it with the WMF's existing device inventory to determine what new devices we need to purchase so that we can be sure we're testing the new features we're developing on the devices the people we are centering in this work are likely to be using on them.

In looking at the Sub-Saharan Africa Editors Supserset dashboard, we see that of the ~2,400 Junior Contributors (<100 cumulative edits) from within SSA who edit the English or French Wikipedias, >80% are living in the ten countries listed below which @Ryasmeen will use to see what devices are popular in these countries so hat we can decide in turn what devices we'll use in the testing we have planned.

I looked at the User Agent information from the Superset dashboard for the SSA countries that we are getting the most edits from, and I listed the devices used here: Google Sheets: T314201. Most of these devices are not available in our test device inventory. Handing this over to you @VPuffetMichel to co-ordinate with the budget owner of test devices to purchase those devices.

I reached out to @Jrbranaa and gave him some information about the EditCheck project. He mentioned that he'll work with @Ryasmeen and @EAkinloose to figure out the devices needed and how to get them for the QTE team.

@Jrbranaa, hi! Are you able to share an update on what – if any – new device support the Editing Team can expect for testing Edit Check-related features?

...I ask the above as the Editing Team gets closer to an initial deployment way of T338907.

Hello @ppelberg, I've just returned to the office. Let me get check on a couple of items and I'll provide a status this coming week (week of July 24th).