Record videos of all Selenium tests
Closed, ResolvedPublic
Actions

Description

Add video recording to wdio-mediawiki's wdio.conf.js file. See T281484: Add a basic wdio.conf.js to wdio-mediawiki.

TODO

T283597: Update wdio-mediawiki to 1.1.1 in all repositories
update Selenium/How-to/Record videos of test runs
Is there a performance difference when screen recording is activated versus not?

Details

Project	Branch	Lines +/-	Subject
mediawiki/core	master	+688 -1 K	selenium: Record videos using Xvfb+ffmpeg
mediawiki/extensions/Cite	master	+876 -434	Selenium: Use wdio-video-reporter for recording test videos
mediawiki/extensions/Math	master	+13 K -387	Selenium: Added wdio-video-reporter for recording test videos
mediawiki/extensions/AbuseFilter	master	+684 -1	Selenium: Use wdio-video-reporter to record videos for Selenium tests
mediawiki/core	master	+86 -62	Selenium: Bump wdio-video-reporter to v3.1.1
mediawiki/extensions/AbuseFilter	master	+349 -3	Selenium: Add video recording to tests

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T210726 Selenium framework cleanup
Resolved	• zeljkofilipin	T281484 Add a basic wdio.conf.js to wdio-mediawiki
Resolved	• zeljkofilipin	T253113 Record videos of all Selenium tests
Declined	Soham	T247960 Move screenshots and video recording from wdio.conf.js to wdio-mediawiki
Resolved	Soham	T252523 Consider wdio-video-reporter as a possible alternative for recording videos from wdio.conf.js
Duplicate	Soham	T255253 Use wdio-video-reporter to record videos in all repositories with Selenium tests
Duplicate	None	T259341 Create a generic NPM package for recording videos of test runs
Resolved	• zeljkofilipin	T276503 Make Echo browser tests record and save videos in CI
Open	None	T284303 ffmpeg exited with code 255
Resolved	• zeljkofilipin	T284335 Remove debugging output when recording videos of Selenium test runs
Open	None	T294341 Remove wdio-video-reporter from all repositories

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

• zeljkofilipin moved this task from Learn/Teach 💡 to Backlog 🪒 on the User-zeljkofilipin board.Aug 31 2020, 11:08 AM

• zeljkofilipin added a parent task: T210726: Selenium framework cleanup.

• zeljkofilipin mentioned this in T210726: Selenium framework cleanup.Aug 31 2020, 11:14 AM

Soham moved this task from Backlog to In Progress on the User-Soham board.Sep 3 2020, 2:43 PM

@Soham I've assigned the task to you, since you've moved it to in-progress column. Feel free to unassign it from yourself, if I misunderstood your action.

Soham mentioned this in T263221: Selenium Framework Cleanup.Sep 18 2020, 3:39 AM

• zeljkofilipin removed Soham as the assignee of this task.Oct 19 2020, 10:10 AM

• zeljkofilipin moved this task from In Progress to Backlog on the User-Soham board.

• zeljkofilipin updated the task description. (Show Details)Oct 19 2020, 3:25 PM

Change 626048 had a related patch set uploaded (by Zfilipin; owner: AlQaholic007):
[mediawiki/core@master] Selenium: Bump wdio-video-reporter to v3.1.1

https://gerrit.wikimedia.org/r/626048

gerritbot added a project: Patch-For-Review.Oct 22 2020, 2:33 PM

• zeljkofilipin moved this task from Backlog 🪒 to In Progress 🎸 on the User-zeljkofilipin board.Oct 22 2020, 2:58 PM

• zeljkofilipin updated the task description. (Show Details)Oct 22 2020, 3:13 PM

Change 605606 had a related patch set uploaded (by Zfilipin; owner: AlQaholic007):
[mediawiki/extensions/AbuseFilter@master] Selenium: Add video recording to tests

https://gerrit.wikimedia.org/r/605606

Change 605606 abandoned by Zfilipin:
[mediawiki/extensions/AbuseFilter@master] Selenium: Add video recording to tests

Reason:

https://gerrit.wikimedia.org/r/605606

• zeljkofilipin updated the task description. (Show Details)Oct 22 2020, 3:23 PM

• zeljkofilipin updated the task description. (Show Details)Oct 22 2020, 3:30 PM

• zeljkofilipin moved this task from In Progress 🎸 to Backlog 🪒 on the User-zeljkofilipin board.Oct 22 2020, 3:32 PM

Change 626048 merged by jenkins-bot:
[mediawiki/core@master] Selenium: Bump wdio-video-reporter to v3.1.1

https://gerrit.wikimedia.org/r/626048

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.16; 2020-11-03).Oct 22 2020, 4:00 PM

santana added a subscriber: santana.Oct 27 2020, 1:10 PM

Richa1496 mentioned this in T266752: Outreachy Project Proposal: Selenium Framework Cleanup .Oct 29 2020, 11:34 AM

Change 637736 had a related patch set uploaded (by Sskenner; owner: Sskenner):
[mediawiki/extensions/AbuseFilter@master] Selenium: Use wdio-video-reporter to record videos for Selenium tests

https://gerrit.wikimedia.org/r/637736

santana updated the task description. (Show Details)Oct 31 2020, 2:14 AM

santana updated the task description. (Show Details)Oct 31 2020, 3:01 AM

• zeljkofilipin moved this task from Backlog 🪒 to In Progress 🎸 on the User-zeljkofilipin board.Nov 4 2020, 1:17 PM

• zeljkofilipin moved this task from In Progress 🎸 to Backlog 🪒 on the User-zeljkofilipin board.Nov 10 2020, 3:22 PM

Vidhi-Mody mentioned this in T274579: Upgrade WebdriverIO to v7 in all repositories.Feb 11 2021, 6:29 PM

• zeljkofilipin claimed this task.Mar 23 2021, 3:03 PM

• zeljkofilipin moved this task from Backlog 🪒 to In Progress 🎸 on the User-zeljkofilipin board.

Change 637736 abandoned by Zfilipin:
[mediawiki/extensions/AbuseFilter@master] Selenium: Use wdio-video-reporter to record videos for Selenium tests

Reason:
@Sskenner: Thanks for working on this. I'm abandoning this patch because we will be using a different tool for recording videos. See T259341.

https://gerrit.wikimedia.org/r/637736

• zeljkofilipin removed • zeljkofilipin as the assignee of this task.Mar 23 2021, 3:19 PM

• zeljkofilipin moved this task from In Progress 🎸 to Backlog 🪒 on the User-zeljkofilipin board.

• zeljkofilipin updated the task description. (Show Details)

Sahilgrewalhere added a subscriber: Sahilgrewalhere.Apr 10 2021, 3:32 PM

Change 678431 had a related patch set uploaded (by Sahilgrewalhere; author: Sahilgrewalhere):

[mediawiki/extensions/Math@master] Selenium: Added wdio-video-reporter for recording test videos

https://gerrit.wikimedia.org/r/678431

Change 678780 had a related patch set uploaded (by Anjali041; author: Anjali041):

[mediawiki/extensions/Cite@master] Selenium: Use wdio-video-reporter for recording test videos

https://gerrit.wikimedia.org/r/678780

Anjali_Kumari_41 added a subscriber: Anjali_Kumari_41.Apr 13 2021, 7:42 AM

Anjali_Kumari_41 mentioned this in T279999: GSoC 2021 Proposal: Upgrade WebdriverIO to the latest version 7 for all repositories.Apr 13 2021, 10:09 AM

Change 681771 had a related patch set uploaded (by Anjali041; author: Anjali041):

[mediawiki/extensions/Cite@master] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Cite into review/anjali041/T253113

https://gerrit.wikimedia.org/r/681771

Change 681771 abandoned by Anjali041:

[mediawiki/extensions/Cite@master] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Cite into review/anjali041/T253113

Reason:

Patch not required

https://gerrit.wikimedia.org/r/681771

• zeljkofilipin added a subtask: T276503: Make Echo browser tests record and save videos in CI.May 4 2021, 5:37 PM

• zeljkofilipin updated the task description. (Show Details)

Out of curiosity, how much disk space would this use? I assume that Jenkins periodically removes old artifacts, does the frequency need to be adjusted depending on the answer to the first question?

As far as I know, Jenkins deletes jobs older than 2 or 4 weeks. I don't think any change is needed.

Core already does record videos for all tests. A recent core patch (683898) triggered wmf-quibble-selenium-php72-docker/92891 job. I've downloaded all files the job generated (archive.zip). The total size of the archive is 42 MB.

I've unpacked the archive and sorted the files by size. mp4 video files are 2-365 KB. png image files are 6-235 KB.

In T253113#7061098, @zeljkofilipin wrote:

As far as I know, Jenkins deletes jobs older than 2 or 4 weeks. I don't think any change is needed.

Core already does record videos for all tests. A recent core patch (683898) triggered wmf-quibble-selenium-php72-docker/92891 job. I've downloaded all files the job generated (archive.zip). The total size of the archive is 42 MB.

I've unpacked the archive and sorted the files by size. mp4 video files are 2-365 KB. png image files are 6-235 KB.

Cool, sounds good!

• zeljkofilipin claimed this task.May 6 2021, 4:38 PM

• zeljkofilipin moved this task from Backlog 🪒 to In Progress 🎸 on the User-zeljkofilipin board.

• zeljkofilipin edited parent tasks, added: T281484: Add a basic wdio.conf.js to wdio-mediawiki; removed: T210726: Selenium framework cleanup.

• zeljkofilipin mentioned this in T281484: Add a basic wdio.conf.js to wdio-mediawiki.May 6 2021, 4:40 PM

Change 678431 abandoned by Zfilipin:

[mediawiki/extensions/Math@master] Selenium: Added wdio-video-reporter for recording test videos

Reason:

Thank you for the patch, but there's a simpler way to solve this. See T281484.

https://gerrit.wikimedia.org/r/678431

Change 678780 abandoned by Zfilipin:

[mediawiki/extensions/Cite@master] Selenium: Use wdio-video-reporter for recording test videos

Reason:

Thank you for the patch, but there's a simpler way to solve this. See T281484.

https://gerrit.wikimedia.org/r/678780

• zeljkofilipin updated the task description. (Show Details)May 6 2021, 4:48 PM

Change 685877 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/core@master] WIP Record videos of all Selenium tests

https://gerrit.wikimedia.org/r/685877

• zeljkofilipin updated the task description. (Show Details)May 6 2021, 5:56 PM

• zeljkofilipin mentioned this in T276503: Make Echo browser tests record and save videos in CI.May 18 2021, 10:59 AM

• zeljkofilipin updated the task description. (Show Details)May 19 2021, 1:50 PM

• zeljkofilipin removed • zeljkofilipin as the assignee of this task.May 20 2021, 1:58 PM

• zeljkofilipin moved this task from In Progress 🎸 to Waiting 🏹 on the User-zeljkofilipin board.

Change 685877 merged by jenkins-bot:

[mediawiki/core@master] selenium: Record videos using Xvfb+ffmpeg

https://gerrit.wikimedia.org/r/685877

Anjali_Kumari_41 removed a subscriber: Anjali_Kumari_41.May 21 2021, 10:48 AM

• zeljkofilipin claimed this task.May 25 2021, 1:32 PM

• zeljkofilipin moved this task from Waiting 🏹 to In Progress 🎸 on the User-zeljkofilipin board.

• zeljkofilipin closed subtask T276503: Make Echo browser tests record and save videos in CI as Resolved.May 27 2021, 9:30 AM

• zeljkofilipin removed • zeljkofilipin as the assignee of this task.May 27 2021, 2:25 PM

• zeljkofilipin updated the task description. (Show Details)

• zeljkofilipin moved this task from In Progress 🎸 to Project ♟ on the User-zeljkofilipin board.

In T253113#7061128, @kostajh wrote:

In T253113#7061098, @zeljkofilipin wrote:

As far as I know, Jenkins deletes jobs older than 2 or 4 weeks. I don't think any change is needed.

Core already does record videos for all tests. A recent core patch (683898) triggered wmf-quibble-selenium-php72-docker/92891 job. I've downloaded all files the job generated (archive.zip). The total size of the archive is 42 MB.

I've unpacked the archive and sorted the files by size. mp4 video files are 2-365 KB. png image files are 6-235 KB.

Cool, sounds good!

Ah, something else I was thinking about – is there a performance difference when screen recording is activated versus not? I assume not but wondering if any benchmarks exist around that. (I've been thinking about slow Selenium a lot more since the main extension I work with was added to the extension-gate, and our CI times went from 5-10 minutes to 25-30 minutes mostly from Selenium tests in other extensions.)

In T253113#7119586, @kostajh wrote:

Ah, something else I was thinking about – is there a performance difference when screen recording is activated versus not? I assume not but wondering if any benchmarks exist around that. (I've been thinking about slow Selenium a lot more since the main extension I work with was added to the extension-gate, and our CI times went from 5-10 minutes to 25-30 minutes mostly from Selenium tests in other extensions.)

Good point. Let me measure that.

• zeljkofilipin updated the task description. (Show Details)Jun 1 2021, 1:33 PM

• zeljkofilipin added a subtask: T284303: ffmpeg exited with code 255.Jun 4 2021, 2:19 PM

In T253113#7119586, @kostajh wrote:

Ah, something else I was thinking about – is there a performance difference when screen recording is activated versus not? I assume not but wondering if any benchmarks exist around that. (I've been thinking about slow Selenium a lot more since the main extension I work with was added to the extension-gate, and our CI times went from 5-10 minutes to 25-30 minutes mostly from Selenium tests in other extensions.)

I ran tests in Fresh container three times.

Without video recording:

nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:44 

real    1m51.320s
user    0m35.715s
sys     0m8.181s

nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:33 

real    1m38.608s
user    0m33.263s
sys     0m7.826s

nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:33 

real    1m40.164s
user    0m35.454s
sys     0m8.038s

• zeljkofilipin added a subtask: T284335: Remove debugging output when recording videos of Selenium test runs.Jun 4 2021, 6:07 PM

With video recording:

nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:56 

real    2m2.450s
user    1m16.099s
sys     0m25.990s

nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:56 

real    2m2.344s
user    1m8.266s
sys     0m19.792s

nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:56 

real    2m2.223s
user    1m9.848s
sys     0m20.622s