Page MenuHomePhabricator

Create performance tests for our most important user journeys
Open, MediumPublic

Description

Today most of our synthetic performance tests is accessing one page with an empty cache. The risk here is that we focus on the wrong things when do not test user journeys. There's also been some blogging about the importance of user journey focus. Today the only user journey we measure is the login user journey.

@Edtadros do you have a list of the most important user journeys that we test on desktop/mobile today? Thinking that could be a starting point for us.

Event Timeline

Jdlrobson triaged this task as Medium priority.
Jdlrobson subscribed.

@Edtadros could you provide this list please?

@Jdlrobson Unfortunately we don't have user journeys (but, we should). Here's my first stab at that list. Let me know if you need more detail.

@Peter, This might be a good thing to discuss with the rest of QTE to see what the other teams are testing. I'm happy to help facilitate.

User Onboarding and Exploration

  • Register a new user account.
  • Log in with the new account.
  • Log in with an existing account
  • Explore the homepage and perform a search.
  • Navigate through various categories and articles.
  • Access and manage user preferences.

Content Discovery and Consumption

  • Perform a search for a specific topic.
  • Access and read several articles on the search results page.
  • View article revisions and related discussions.
  • Navigate to related articles through internal links.
  • Download files or view media associated with the articles.

Content Creation and Collaboration

  • Access and edit an article.
  • Save changes and view the updated article.
  • Review and manage revisions and talk pages.
  • Use real-time collaboration features (if applicable).
  • Monitor notifications for any feedback or changes.

User Interaction and Communication

  • Send and receive messages within the platform.
  • Access and manage notifications.
  • View and respond to discussion or talk page comments.
  • Access and manage the watchlist.

Thank you @Edtadros let's discuss it the next time we meet!

One thing about the user journeys: I think it's important that we can identify which team is responsible for each journey so we don't create the tests and they will not be used.

By responsible I mean: 1. Who will fix the script if the test stops to work. 2. Who will act on a performance regression in that area (for example, an alert is fired, who will investigate the regression).

We're talking about this in next steering meeting.

@Peter assigning this back to you so that it doesn't fall off your radar. Put something on my calendar if you'd like to meet.

Thanks @Edtadros I'll try to find a time that works for us both next week, will send an invite.

Jdlrobson-WMF subscribed.

Hey @Edtadros @Peter the engineers discussed this today but weren't sure how to make it actionable. Will this perhaps be discussed as part of T373169: Iterate and update Web Team Site Performance dashboard ?

this today but weren't sure how to make it actionable.

I think the first step is to identify which are our most important user journeys, for me this is something that product/product managers can help with. Then when we have the 5 most important ones, lets discuss which metrics should be important for each journey.

For example, say that a searching from the header is identified as a top 5 journey. Then we add a synthetic test that do a search and we measure the time from when the users click on search and the content become visible. And then best case we can also add that metric to our real user measurement. We can then track these and make sure we don't get any regressions in these metrics.

Does that sound ok?