Page MenuHomePhabricator

Record videos of all Selenium tests
Open, MediumPublic

Description

Add video recording to wdio-mediawiki's wdio.conf.js file. See T281484: Add a basic wdio.conf.js to wdio-mediawiki.

Related commit: eed975d6e356e3f5c2df1b79265d14c8fb3e38c6 604635

TODO

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Soham updated the task description. (Show Details)
zeljkofilipin moved this task from In Progress to Waiting/Blocked on the User-Soham board.

@Soham I've assigned the task to you, since you've moved it to in-progress column. Feel free to unassign it from yourself, if I misunderstood your action.

zeljkofilipin moved this task from In Progress to Backlog on the User-Soham board.

Change 626048 had a related patch set uploaded (by Zfilipin; owner: AlQaholic007):
[mediawiki/core@master] Selenium: Bump wdio-video-reporter to v3.1.1

https://gerrit.wikimedia.org/r/626048

Change 605606 had a related patch set uploaded (by Zfilipin; owner: AlQaholic007):
[mediawiki/extensions/AbuseFilter@master] Selenium: Add video recording to tests

https://gerrit.wikimedia.org/r/605606

Change 605606 abandoned by Zfilipin:
[mediawiki/extensions/AbuseFilter@master] Selenium: Add video recording to tests

Reason:

https://gerrit.wikimedia.org/r/605606

Change 626048 merged by jenkins-bot:
[mediawiki/core@master] Selenium: Bump wdio-video-reporter to v3.1.1

https://gerrit.wikimedia.org/r/626048

Change 637736 had a related patch set uploaded (by Sskenner; owner: Sskenner):
[mediawiki/extensions/AbuseFilter@master] Selenium: Use wdio-video-reporter to record videos for Selenium tests

https://gerrit.wikimedia.org/r/637736

Change 637736 abandoned by Zfilipin:
[mediawiki/extensions/AbuseFilter@master] Selenium: Use wdio-video-reporter to record videos for Selenium tests

Reason:
@Sskenner: Thanks for working on this. I'm abandoning this patch because we will be using a different tool for recording videos. See T259341.

https://gerrit.wikimedia.org/r/637736

Change 678431 had a related patch set uploaded (by Sahilgrewalhere; author: Sahilgrewalhere):

[mediawiki/extensions/Math@master] Selenium: Added wdio-video-reporter for recording test videos

https://gerrit.wikimedia.org/r/678431

Change 678780 had a related patch set uploaded (by Anjali041; author: Anjali041):

[mediawiki/extensions/Cite@master] Selenium: Use wdio-video-reporter for recording test videos

https://gerrit.wikimedia.org/r/678780

Change 681771 had a related patch set uploaded (by Anjali041; author: Anjali041):

[mediawiki/extensions/Cite@master] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Cite into review/anjali041/T253113

https://gerrit.wikimedia.org/r/681771

Change 681771 abandoned by Anjali041:

[mediawiki/extensions/Cite@master] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Cite into review/anjali041/T253113

Reason:

Patch not required

https://gerrit.wikimedia.org/r/681771

Out of curiosity, how much disk space would this use? I assume that Jenkins periodically removes old artifacts, does the frequency need to be adjusted depending on the answer to the first question?

As far as I know, Jenkins deletes jobs older than 2 or 4 weeks. I don't think any change is needed.

Core already does record videos for all tests. A recent core patch (683898) triggered wmf-quibble-selenium-php72-docker/92891 job. I've downloaded all files the job generated (archive.zip). The total size of the archive is 42 MB.

I've unpacked the archive and sorted the files by size. mp4 video files are 2-365 KB. png image files are 6-235 KB.

As far as I know, Jenkins deletes jobs older than 2 or 4 weeks. I don't think any change is needed.

Core already does record videos for all tests. A recent core patch (683898) triggered wmf-quibble-selenium-php72-docker/92891 job. I've downloaded all files the job generated (archive.zip). The total size of the archive is 42 MB.

I've unpacked the archive and sorted the files by size. mp4 video files are 2-365 KB. png image files are 6-235 KB.

Cool, sounds good!

Change 678431 abandoned by Zfilipin:

[mediawiki/extensions/Math@master] Selenium: Added wdio-video-reporter for recording test videos

Reason:

Thank you for the patch, but there's a simpler way to solve this. See T281484.

https://gerrit.wikimedia.org/r/678431

Change 678780 abandoned by Zfilipin:

[mediawiki/extensions/Cite@master] Selenium: Use wdio-video-reporter for recording test videos

Reason:

Thank you for the patch, but there's a simpler way to solve this. See T281484.

https://gerrit.wikimedia.org/r/678780

Change 685877 had a related patch set uploaded (by Zfilipin; author: Zfilipin):

[mediawiki/core@master] WIP Record videos of all Selenium tests

https://gerrit.wikimedia.org/r/685877

Change 685877 merged by jenkins-bot:

[mediawiki/core@master] selenium: Record videos using Xvfb+ffmpeg

https://gerrit.wikimedia.org/r/685877

zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)

As far as I know, Jenkins deletes jobs older than 2 or 4 weeks. I don't think any change is needed.

Core already does record videos for all tests. A recent core patch (683898) triggered wmf-quibble-selenium-php72-docker/92891 job. I've downloaded all files the job generated (archive.zip). The total size of the archive is 42 MB.

I've unpacked the archive and sorted the files by size. mp4 video files are 2-365 KB. png image files are 6-235 KB.

Cool, sounds good!

Ah, something else I was thinking about โ€“ is there a performance difference when screen recording is activated versus not? I assume not but wondering if any benchmarks exist around that. (I've been thinking about slow Selenium a lot more since the main extension I work with was added to the extension-gate, and our CI times went from 5-10 minutes to 25-30 minutes mostly from Selenium tests in other extensions.)

Ah, something else I was thinking about โ€“ is there a performance difference when screen recording is activated versus not? I assume not but wondering if any benchmarks exist around that. (I've been thinking about slow Selenium a lot more since the main extension I work with was added to the extension-gate, and our CI times went from 5-10 minutes to 25-30 minutes mostly from Selenium tests in other extensions.)

Good point. Let me measure that.

Ah, something else I was thinking about โ€“ is there a performance difference when screen recording is activated versus not? I assume not but wondering if any benchmarks exist around that. (I've been thinking about slow Selenium a lot more since the main extension I work with was added to the extension-gate, and our CI times went from 5-10 minutes to 25-30 minutes mostly from Selenium tests in other extensions.)

I ran tests in Fresh container three times.

Without video recording:

nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:44 

real    1m51.320s
user    0m35.715s
sys     0m8.181s
nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:33 

real    1m38.608s
user    0m33.263s
sys     0m7.826s
nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:33 

real    1m40.164s
user    0m35.454s
sys     0m8.038s

With video recording:

nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:56 

real    2m2.450s
user    1m16.099s
sys     0m25.990s
nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:56 

real    2m2.344s
user    1m8.266s
sys     0m19.792s
nobody@docker-desktop:/core$ time npm run selenium-test
...
Spec Files:      5 passed, 5 total (100% completed) in 00:01:56 

real    2m2.223s
user    1m9.848s
sys     0m20.622s

Tests are slower when recording videos. I think the benefit of having videos is worth the slowdown.