Write a suite of user experience end-to-end integration tests for WikiLambda in selenium/etc.
Open, HighPublic
Actions

Description

We should have a framework for integration tests, that ensure the different components work together end-to-end. That includes the front end, the wiki, and the evaluator (orchestrator and executor).

Details

Project	Branch	Lines +/-	Subject
mediawiki/extensions/WikiLambda	master	+115 -1	Write a suite of user experience end-to-end integration tests
mediawiki/extensions/WikiLambda	master	+406 -54	selenium: Install missing npm dependencies
mediawiki/extensions/WikiLambda	master	+4 K -3 K	Add initial selenium tests

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T294388 Run some full end-to-end browser integration tests on Beta Cluster Wikifunctions now it exists
Open	SDunlap	T276847 Write a suite of user experience end-to-end integration tests for WikiLambda in selenium/etc.
Resolved	Teleosteen	T321932 E2E test for CUJ 1: Evaluate a function
Open	Teleosteen	T321933 E2E test for CUJ 2: Create a function definition
Resolved	DVrandecic	T318930 CUJ 2: Create a function definition
Resolved	DVrandecic	T323804 Design: create a function definition
Resolved	Ddwaal	T321399 Don't show / disable the "show more languages" / "Show name in other languages" buttons when there aren't any to show
Resolved	Ddwaal	T321396 Copy: Evaluate copy on function viewer details tab
Open	Nik.xyz.in	T321934 E2E test for CUJ 3: Edit a function definition
Resolved	DVrandecic	T318933 CUJ 3: Edit a function definition
Resolved	DVrandecic	T318314 Function editor: Remove an item from the list of arguments
Resolved	• AdesojiThisDot	T318315 Function editor: Implement functionality of "remove input field" using the Mobile view button.
Resolved	Ddwaal	T318317 Function editor: Implement "remove input field" button for Desktop view
Resolved	AAlhazwani-WMF	T312121 The dropdown menu of the "output type" from the function definition page is hidden under another component
Resolved	None	T323805 Design: edit a function definition
Resolved	aishwaryavardhana	T285418 A way to easily and intuitively create and edit functions
Resolved	aishwaryavardhana	T285419 Initial user flow diagrams for function editor
Invalid	aishwaryavardhana	T285421 Generative design research: Testing of new hypotheses and mental models
Declined	SWoodruff-WMF	T285422 Function editor: User education, interface copy, notifications and alerts
Declined	SWoodruff-WMF	T285684 As someone who is new to creating functions, I would like to be able to start with an easy task
Resolved	aishwaryavardhana	T296693 Desktop designs for function editor v1
Resolved	• AdesojiThisDot	T297086 Function Editor - Sidebar - Language selector
Declined	• AdesojiThisDot	T298203 Provide an easy way on the front-end for Wikifunctions to know what languages are supported (and what their ZIDs are)
Resolved	SimoneThisDot	T297095 Function Editor - Function Definition - New component
Resolved	JKieserman	T297098 Function Editor - Function definition - Heading
Resolved	• AdesojiThisDot	T297099 Function Editor - Enhance the "Object selector" to allow user to type within its first entry instead than the input itself to align with the designs
Resolved	SimoneThisDot	T297100 Function Editor - Create the function definition main content area
Resolved	• AdesojiThisDot	T297101 Function Editor - Allow user to add labels in another language in function definition
Resolved	JKieserman	T297104 Function Editor - Provide the ability for users to publish, write implementation and cancel function
Resolved	JKieserman	T297271 Function Editor - Create an Chip component
Resolved	• AdesojiThisDot	T297280 Function Editor - sidebar - Update diagram
Resolved	gengh	T297869 Function Editor: Allow user to REMOVE an Input from a function definition
Resolved	SimoneThisDot	T298464 Remove old function editor and its files
Resolved	aishwaryavardhana	T296698 Provide a UX design for function viewing
Resolved	SimoneThisDot	T301518 Explore replacing the legacy function editor with fallback to the generic editor via a button, so that functions can be connected to implementations and testers and we can scrap the legacy function editor
Resolved	aishwaryavardhana	T296694 Mobile designs for function editor v1
Resolved	aishwaryavardhana	T296696 Provide a compelling, scalable, validated UX design for function creation and editing
Resolved	aishwaryavardhana	T297458 User flow for when a user edits definition
Resolved	aishwaryavardhana	T297462 User flow when a user only edits labels
Resolved	aishwaryavardhana	T297463 User flow for how a user would complete multilingual edits
Resolved	None	T297464 Decide on the UX design for the edit summary
Resolved	aishwaryavardhana	T297465 Design notes for if an input type is changed
Resolved	aishwaryavardhana	T297466 Design notes for if an output type is changed
Resolved	aishwaryavardhana	T297467 Decide on the UX design for attaching and detatching (but not editing) implementations and testers
Resolved	aishwaryavardhana	T297668 Revisit user flows for function editor
Resolved	SWoodruff-WMF	T318316 Function editor: Design function argument (input) delete button/icon for Desktop view.
Resolved	AAlhazwani-WMF	T320410 Design: Function editor, redefine separation between fields and blocks.
Resolved	AAlhazwani-WMF	T270304 Design a "Publish" component
Resolved	SimoneThisDot	T268296 Display to user why an object is not stored
Duplicate	None	T274424 Show errors if creation of ZObject fails
Resolved	DVrandecic	T316686 Copy: change text for confirmation dialog when creating a new function on mobile
Resolved	DVrandecic	T303336 Users can write an edit summary when publishing which can be seen by other users in histories
Resolved	DVrandecic	T299004 Have the publish function enable/disable itself only if there have been changes
Resolved	gengh	T326656 FunctionEditor: Implement separation between fields and blocks changes
Open	Nik.xyz.in	T321935 E2E test for CUJ 4: Create tester for function
Open	None	T318938 CUJ 4: Create tests
Open	None	T323806 Design: create tests
Open	Sushrith_Bogi	T321937 E2E test for CUJ 5: Create implementations with code
Open	None	T318939 CUJ 5: Add implementations via code or composition
Invalid	None	T323807 Design: add implementations via code or composition
Open	Sushrith_Bogi	T321939 E2E test for CUJ 6: Connect an implementation or test to a function
Resolved	DVrandecic	T318936 CUJ 6: Connect an implementation or test to a function
Resolved	DVrandecic	T323808 Design: connect an implementation or test to a function
Resolved	Ddwaal	T320278 Copy: Language for "connecting" an implementation or tester to a ZFunction
Resolved	Ddwaal	T318426 Tests: Implement FE integration test for connecting implementations and testers to functions
Open	None	T321940 E2E test for CUJ 7: Create and edit types
Open	None	T318943 CUJ 7: Create and edit types
Open	None	T323809 Design: create and edit types
Resolved	AAlhazwani-WMF	T306442 Design the fallback view experience for non-function ZObjects
Resolved	gengh	T323013 Default component: nested logic and visual style
Open	SDunlap	T321942 make E2E test suit runnable via "helm test"
Open	None	T330702 Extend the CUJ1 browser test asserting that a function can be called and returns a result to ensure we can adjust input values

Event Timeline

DVrandecic created this task.Mar 8 2021, 5:59 PM

Jdforrester-WMF renamed this task from Integration tests to Write a suite of user experience end-to-end integration tests for WikiLambda in selenium/etc..Mar 8 2021, 6:55 PM

DVrandecic moved this task from To triage to Phase ζ on the Abstract Wikipedia team board.Mar 10 2021, 5:21 PM

DVrandecic edited projects, added Abstract Wikipedia team (Phase ζ); removed Abstract Wikipedia team.

Change 697056 had a related patch set uploaded (by Simone Cuomo; author: Simone Cuomo):

[mediawiki/extensions/WikiLambda@master] Add selenium to Repository

https://gerrit.wikimedia.org/r/697056

gerritbot added a project: Patch-For-Review.May 28 2021, 1:23 PM

zeljkofilipin added a project: User-zeljkofilipin.May 28 2021, 1:58 PM

zeljkofilipin moved this task from Backlog 🪒 to Watching 📺 on the User-zeljkofilipin board.

I've left a couple of comments on the patch. Let me know if you need any help.

Sahilgrewalhere added a subscriber: Sahilgrewalhere.May 31 2021, 1:37 PM

Change 697056 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Add initial selenium tests

https://gerrit.wikimedia.org/r/697056

Maintenance_bot removed a project: Patch-For-Review.Jun 1 2021, 9:10 PM

DVrandecic moved this task from Phase ζ to Un-phased work on the Abstract Wikipedia team board.Jun 2 2021, 4:42 PM

DVrandecic edited projects, added Abstract Wikipedia team; removed Abstract Wikipedia team (Phase ζ).

To run a full end-to-end test in CI, we'd need T280070: Provide a CI mechanism by which we can test end-to-end integration between MW and back-end services, e.g. via docker-compose to be resolved first.

DVrandecic assigned this task to SimoneThisDot.Jun 9 2021, 4:43 PM

DVrandecic triaged this task as High priority.

Change 699736 had a related patch set uploaded (by Simone Cuomo; author: Simone Cuomo):

[mediawiki/extensions/WikiLambda@master] Write a suite of user experience end-to-end integration tests

https://gerrit.wikimedia.org/r/699736

gerritbot added a project: Patch-For-Review.Jun 14 2021, 11:57 AM

Change 702120 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/extensions/WikiLambda@master] selenium: Install missing npm dependencies

https://gerrit.wikimedia.org/r/702120

Change 702120 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] selenium: Install missing npm dependencies

https://gerrit.wikimedia.org/r/702120

zeljkofilipin moved this task from Watching 📺 to In Progress 🎸 on the User-zeljkofilipin board.Jul 2 2021, 1:29 PM

Lindsaykwardell mentioned this in T286228: Expand E2E test coverage.Aug 4 2021, 5:33 PM

Jdforrester-WMF merged a task: T286228: Expand E2E test coverage.Aug 4 2021, 5:53 PM

Jdforrester-WMF added subscribers: SimoneThisDot, Lindsaykwardell.

In T276847#7129612, @Jdforrester-WMF wrote:

To run a full end-to-end test in CI, we'd need T280070: Provide a CI mechanism by which we can test end-to-end integration between MW and back-end services, e.g. via docker-compose to be resolved first.

Is this task still blocked?

zeljkofilipin moved this task from In Progress 🎸 to Watching 📺 on the User-zeljkofilipin board.Aug 24 2021, 2:22 PM

To the best of my knowledge, yes.

Jdforrester-WMF removed SimoneThisDot as the assignee of this task.Sep 14 2021, 8:17 PM

Jdforrester-WMF edited projects, added Abstract Wikipedia team (Phase κ – Clean-up); removed Abstract Wikipedia team.

zeljkofilipin added a project: Browser-Tests.Oct 7 2021, 11:12 AM

SDunlap claimed this task.Oct 17 2022, 3:17 PM

Jdforrester-WMF moved this task from Incoming to Blocking launch on the Abstract Wikipedia team (Phase κ – Clean-up) board.Oct 24 2022, 8:24 PM

Jdforrester-WMF added a parent task: T294388: Run some full end-to-end browser integration tests on Beta Cluster Wikifunctions now it exists.Oct 25 2022, 1:43 PM

SDunlap mentioned this in T321405: Proposal: end-to-end tests with Gherkin syntax and automated browsers.Oct 25 2022, 9:37 PM

SDunlap merged a task: T321405: Proposal: end-to-end tests with Gherkin syntax and automated browsers.Oct 28 2022, 6:38 PM

SDunlap added subscribers: Teleosteen, SDunlap.