Page MenuHomePhabricator
Create Task
Maniphest T227071

Identify and Develop crucial paths e2e tests in Wikibase
Closed, DuplicatePublic
Actions

Description

AC

  • Explore the different paths our users take to achieve tasks on our software (use-cases/jobs-to-be-done), and document how many of these were explored in depth to sort out the critical ones.
  • Conclude a set of crucial paths in Wikibase interfaces (web ui or api) from the previous step:

    A crucial path is one that a user take for a use-case when they are using Wikibase. The following are only examples:
    • readers want to be able to view and item/property, navigate through the links provided in the response and do on web ui (with and without js)
    • editors want to be able to create a new item on web ui (with and without js)
    • editors want to be able to propose a new property on web ui (with and without js)
    • editors want to be able to add/update statements on an item/property, and add qualifiers/references to them on web ui (with and without js)
    • bots want to be able to query entities through the api
    • bots want to be able to create items through the api
    • bots want to be able to add/update statements on an item/property, and add qualifiers/references to them through the api
  • end-to-end tests for these crucial paths are implemented in the form of:
    • browser tests for web ui paths
    • api tests for api paths

Notes

  • those tests should avoid depending on interfaces on the same level. For example, browser tests should avoid using the api to prepare the backend system for their tests. The backend system should be initialized for each test with appropriate fixtures.

Related Objects
Search...

Event Timeline

• alaa_wmde created this task.Jul 2 2019, 9:48 AM
• alaa_wmde added a subscriber: Addshore.EditedJul 2 2019, 9:50 AM

@Addshore I'd like to put this a trailblazing in the space we have until 01.08. To be followed later with more sub-tasks on the parent one in next trailblazings that focus on developer productivity ( + maintenance on our pipeline as browser tests are causing pain recently)

• alaa_wmde updated the task description. (Show Details)Jul 2 2019, 9:52 AM
• alaa_wmde moved this task from Backlog to In Investigation (cost/impact) on the Wikidata-Trailblazing-Exploration board.Jul 2 2019, 10:28 AM
Lydia_Pintscher added a project: Wikidata.Jul 2 2019, 2:15 PM
Addshore added a comment.Jul 9 2019, 4:23 PM
In T227071#5299460, @alaa_wmde wrote:

@Addshore I'd like to put this a trailblazing in the space we have until 01.08. To be followed later with more sub-tasks on the parent one in next trailblazings that focus on developer productivity ( + maintenance on our pipeline as browser tests are causing pain recently)

I'd be interested to see the outcomes.

My only thought is that "Small set of crucial paths" could result in the list being rather incomplete.
Of course a complete list would be most interesting.
Perhaps we should add to the AC that the trailblazers should try to identify what proportion of the paths they have probably looked at and documented?

It would also be good to clarify when we are going to run these e2e tests?

Also we currently have a hard enough time staying on top of the current sort of e2e tests, mainly browser tests.
Is any part of this going to remove browser tests in favor of some of these e2e tests, or will we just end up having more tests / more layers of tests?

• alaa_wmde added a comment.Jul 24 2019, 8:54 PM

My only thought is that "Small set of crucial paths" could result in the list being rather incomplete.
Of course a complete list would be most interesting.
Perhaps we should add to the AC that the trailblazers should try to identify what proportion of the paths they have probably looked at and documented?

Sounds good will add something to AC. I understand from this that the concern is that we don't really cover all the critical e2e paths in the first run. We can try to collect as many as we can, but probably we will have to add another one or two (or more) in the future as we discover them.

It would also be good to clarify when we are going to run these e2e tests?

Yeap that should definitely part of the trailblazing outcome. I can imagine it being the last type of tests to run in our build (unit -> integraiton -> e2e). But there could be surely more to it than that once we get to the details.

Also we currently have a hard enough time staying on top of the current sort of e2e tests, mainly browser tests.
Is any part of this going to remove browser tests in favor of some of these e2e tests, or will we just end up having more tests / more layers of tests?

Ideally, I think we would probably run those e2e (browser and api tests) along the rest of our browser tests until we are happy with their stability.. the next step, as in the parent's description is actually to start removing browsers tests that are not necessary, maybe by converting some of them to unit and integration tests on frontends and backends.

• alaa_wmde updated the task description. (Show Details)Jul 26 2019, 9:54 AM
• alaa_wmde added a comment.Jul 30 2019, 2:22 PM

Let's break this down, by interface (API/UI) so that we have smaller iterations to achieve the first AC point.

Addshore mentioned this in T234636: Wikimedia Technical Conference 2019 Session: API Integration Testing.Oct 31 2019, 7:09 AM
Addshore added a comment.Nov 12 2019, 3:32 PM

So, first thing we should look at (coming from tech conf 2019) https://www.mediawiki.org/wiki/MediaWiki_API_Integration_Tests

Addshore moved this task from In Investigation (cost/impact) to Backlog on the Wikidata-Trailblazing-Exploration board.Jan 29 2020, 11:28 AM
Addshore added a project: [DEPRECATED] wdwb-tech.Jan 22 2021, 10:03 PM
Addshore closed subtask T229334: Explore user paths through client UI and identify cruicial ones as Resolved.Jan 22 2021, 10:25 PM
Addshore closed subtask T229335: Explore user paths through API and identify cruicial ones as Resolved.
Addshore added subscribers: • toan, Tarrow, hoo.

@Tarrow @toan @hoo Do you think this is happening somewhat as part of the release strategy? (Perhaps at a higher level?)

Tarrow added a comment.Jan 22 2021, 10:29 PM
In T227071#6770344, @Addshore wrote:

@Tarrow @toan @hoo Do you think this is happening somewhat as part of the release strategy? (Perhaps at a higher level?)

It is indeed; somewhat happening. See T272414

Addshore closed this task as a duplicate of T272414: Create automated tests ensuring that Wikibase software package work correctly in all environments communicated as supported/compatible.Jan 22 2021, 11:14 PM

I'm going to let that become the first iteration of this task / idea

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL