Page MenuHomePhabricator
Create Task
Maniphest T298882

Selenium Tests failing for CLI on Mac
Closed, ResolvedPublic
Actions

Description

System Information for MAC
Operating System: MacOS Monterey (12.1)
Node Version: v17.1.0
NPM version: 8.1.2

How to reproduce the issue?
After following the steps mentioned for CLI and updating, I followed the docker documentation. While being in the core folder, I set up the environment variables for CLI and did the following commands

fresh-node -env -net
npm ci
npm run selenium-test

Observation and Error description
I was able to get the following error mentioned on the MAC while running the selenium test

Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)

Further description can be seen at P18420.

System Information for Linux

Operating System: Ubuntu 64-bit 20.04.3
Node Version: v17.1.0
NPM version: 8.1.2

Observation
On Linux, it is working fine, and all tests were passing. For further information on Linux, it can be seen at P18422.

Related Objects

Event Timeline

Osamaahmed17 created this task.Jan 10 2022, 1:19 PM
zeljkofilipin added a project: User-zeljkofilipin.Jan 10 2022, 1:46 PM
zeljkofilipin moved this task from Backlog 🪒 to Waiting 🏹 on the User-zeljkofilipin board.
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin added a comment.Jan 10 2022, 1:49 PM

@Osamaahmed17 is an Outreachy intern working on T256626: Refactor WebdriverIO tests from sync to async mode. I've noticed the same problem. Running Selenium tests using cli/docker works just fine on linux, but it fails on macos.

zeljkofilipin added a comment.Jan 10 2022, 2:37 PM

Both on macos and linux, the local wiki seems to work fine, when tested manually.

Addshore moved this task from Inbox to Backlog on the mwcli board.Jan 12 2022, 9:55 AM
Addshore subscribed.

The error makes it look like it is something to do with domain name resolution?
Can you resolve / curl the wiki / site from inside one of the containers running as part of mwcli?

zeljkofilipin moved this task from Waiting 🏹 to Deep work 🌊 on the User-zeljkofilipin board.Jan 12 2022, 1:48 PM
zeljkofilipin moved this task from Deep work 🌊 to Waiting 🏹 on the User-zeljkofilipin board.Jan 12 2022, 3:38 PM
Osamaahmed17 added a comment.Jan 13 2022, 4:34 PM

Hi @Addshore, I didn't know what you mean but I guess, you were referring to curling the wiki. Is this what you meant?

osamaahmedtahir@Users-MacBook-Pro core % curl --version
curl 7.77.0 (x86_64-apple-darwin21.0) libcurl/7.77.0 (SecureTransport) LibreSSL/2.8.3 zlib/1.2.11 nghttp2/1.42.0
Release-Date: 2021-05-26
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets
osamaahmedtahir@Users-MacBook-Pro core % curl -v http://default.mediawiki.mwdd.localhost:8080 
*   Trying 127.0.0.1:8080...
* Connected to default.mediawiki.mwdd.localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: default.mediawiki.mwdd.localhost:8080
> User-Agent: curl/7.77.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.21.1
< Date: Thu, 13 Jan 2022 16:27:52 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 0
< Connection: keep-alive
< X-Content-Type-Options: nosniff
< Vary: Accept-Encoding,Cookie
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< Cache-Control: private, must-revalidate, max-age=0
< X-Request-Id: 34efaf1abadd46af26976dfe
< Upgrade: h2
< Last-Modified: Thu, 13 Jan 2022 16:27:52 GMT
< Location: http://default.mediawiki.mwdd.localhost:8080/w/index.php?title=Main_Page
< 
* Connection #0 to host default.mediawiki.mwdd.localhost left intact
Osamaahmed17 added a comment.Jan 13 2022, 4:39 PM

Plus these are the following containers running on my machine and do you mean to curl from one of those containers?

osamaahmedtahir@Users-MacBook-Pro core % docker ps
CONTAINER ID   IMAGE                                                       COMMAND                  CREATED          STATUS          PORTS                            NAMES
2329812f3ad0   docker-registry.wikimedia.org/dev/stretch-php73-fpm:3.0.0   "/mwdd/entrypoint.sh"    12 minutes ago   Up 12 minutes   9000/tcp                         mwcli-mwdd-default_mediawiki_1
da8747f0b59c   docker-registry.wikimedia.org/dev/stretch-apache2:2.0.0     "/bin/bash /entrypoi…"   12 minutes ago   Up 12 minutes   8080/tcp                         mwcli-mwdd-default_mediawiki-web_1
e0040db74aa9   jwilder/nginx-proxy:0.9                                     "/app/docker-entrypo…"   12 minutes ago   Up 12 minutes   80/tcp, 0.0.0.0:8080->8080/tcp   mwcli-mwdd-default_nginx-proxy_1
8f8657681e65   defreitas/dns-proxy-server:2.19.0                           "/app/dns-proxy-serv…"   12 minutes ago   Up 12 minutes                                    mwcli-mwdd-default_dps_1
Osamaahmed17 added a comment.Jan 13 2022, 4:48 PM

After running the curl command from one of the containers named mwcli-mwdd-default_mediawiki-web_1, I was able to get the following response.

osamaahmedtahir@Users-MacBook-Pro ~ % docker exec -it da8747f0b59c38897713c74cb29a232448cd5aabe9cecffe9a15eb189efc9d72 /bin/sh
# curl --version
curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.2u zlib/1.2.8 libidn2/0.16 libpsl/0.17.0 (+libidn2/0.16) libssh2/1.7.0 nghttp2/1.18.1 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL
# curl -v http://default.mediawiki.mwdd.localhost:8080 
* Rebuilt URL to: http://default.mediawiki.mwdd.localhost:8080/
*   Trying 10.0.0.2...
* TCP_NODELAY set
* Connected to default.mediawiki.mwdd.localhost (10.0.0.2) port 8080 (#0)
> GET / HTTP/1.1
> Host: default.mediawiki.mwdd.localhost:8080
> User-Agent: curl/7.52.1
> Accept: */*
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.21.1
< Date: Thu, 13 Jan 2022 16:43:50 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 0
< Connection: keep-alive
< X-Content-Type-Options: nosniff
< Vary: Accept-Encoding,Cookie
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< Cache-Control: private, must-revalidate, max-age=0
< X-Request-Id: d8e90a8bce1cd6757c92efd2
< Upgrade: h2
< Last-Modified: Thu, 13 Jan 2022 16:43:50 GMT
< Location: http://default.mediawiki.mwdd.localhost:8080/w/index.php?title=Main_Page
< 
* Curl_http_done: called premature == 0
* Connection #0 to host default.mediawiki.mwdd.localhost left intact
Addshore added a comment.Jan 13 2022, 4:51 PM

Thaks, so this looks like the mediawiki host is working correctly both inside and outside containers, as both curl show you the 301 redirect to the main page.

Specifically what command are you using to run the tests?
And which test / tests?

Osamaahmed17 added a comment.Jan 13 2022, 5:09 PM

Hi @Addshore, Thank you for the quick reply! Well currently, I am running selenium tests.

osamaahmedtahir@Users-MacBook-Pro core % npm --version        
7.24.2
osamaahmedtahir@Users-MacBook-Pro core % node --version
v14.18.3
osamaahmedtahir@Users-MacBook-Pro core % export MW_SERVER=http://default.mediawiki.mwdd.localhost:8080
export MW_SCRIPT_PATH=/w
export MEDIAWIKI_USER=admin
export MEDIAWIKI_PASSWORD=mwddpassword
osamaahmedtahir@Users-MacBook-Pro core % npm run selenium-test

> selenium-test
> wdio ./tests/selenium/wdio.conf.js

Execution of 5 spec files started at 2022-01-13T17:04:45.474Z

[0-0] RUNNING in chrome - /tests/selenium/specs/page.js
[0-1] RUNNING in chrome - /tests/selenium/specs/recentchanges.js
[0-2] RUNNING in chrome - /tests/selenium/specs/user.js
[0-3] RUNNING in chrome - /tests/selenium/specs/watchlist.js
[0-0] Error in "Page."before all" hook for "should be previewable""
Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:71:26)
[0-1] Error in "Special:RecentChanges."before all" hook for "shows page creation""
Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:71:26)
[0-3] Error in "Special:Watchlist."before all" hook for "should show page with new edit""
Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:71:26)
[0-2] Error in "User."before all" hook for "should be able to create account""
Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:71:26)
[0-0] RETRYING in chrome - /tests/selenium/specs/page.js
[0-1] RETRYING in chrome - /tests/selenium/specs/recentchanges.js
[0-2] RETRYING in chrome - /tests/selenium/specs/user.js
[0-3] RETRYING in chrome - /tests/selenium/specs/watchlist.js
[0-1] RUNNING in chrome - /tests/selenium/specs/recentchanges.js
[0-2] RUNNING in chrome - /tests/selenium/specs/user.js
[0-0] RUNNING in chrome - /tests/selenium/specs/page.js
[0-3] RUNNING in chrome - /tests/selenium/specs/watchlist.js
[0-3] Error in "Special:Watchlist."before all" hook for "should show page with new edit""
Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:71:26)
[0-1] Error in "Special:RecentChanges."before all" hook for "shows page creation""
Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:71:26)
[0-0] Error in "Page."before all" hook for "should be previewable""
Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:71:26)
[0-2] Error in "User."before all" hook for "should be able to create account""
Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:71:26)
[0-0] FAILED in chrome - /tests/selenium/specs/page.js (1 retries)
[0-1] FAILED in chrome - /tests/selenium/specs/recentchanges.js (1 retries)
[0-2] FAILED in chrome - /tests/selenium/specs/user.js (1 retries)
[0-3] FAILED in chrome - /tests/selenium/specs/watchlist.js (1 retries)
[0-4] RUNNING in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js
[0-4] PASSED in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js

 "dot" Reporter:
.

Spec Files:      1 passed, 4 retries, 4 failed, 5 total (100% completed) in 00:00:18
Addshore added a comment.Jan 13 2022, 5:11 PM

Please try mw docker mediawiki fresh npm run selenium-test

The docs for the fresh command are

Examples:
        # Start an interactive terminal in the fresh container
        fresh bash

        # Run npm ci in the currently directory (if within mediawiki)
        fresh npm ci                                    # Run npm ci in the current directory (if within mediawiki)

        # Run mediawiki core tests (when in the mediawiki core directory)
        fresh npm run selenium-test

        # Run a single Wikibase extension test spec (when in the Wikibase extension directory)
        fresh npm run selenium-test:repo -- -- --spec repo/tests/selenium/specs/item.js
Addshore added a comment.Jan 13 2022, 5:13 PM

Error: getaddrinfo ENOTFOUND default.mediawiki.mwdd.localhost from your local machine sounds like you are not able to resolve the domain

I expect you also can not ping that host?

See also the mw docker hosts command for altering your hosts file to make these resolve on tools such as various CLI things.
Most browsers now by default resolve *.localhost domains to 127.0.0.1, but many other tools dont.

Osamaahmed17 added a comment.Jan 13 2022, 9:38 PM

Hi @Addshore, after running the mw docker mediawiki fresh npm run selenium-test, the selenium tests were successful. Here are the results mentioned below.

osamaahmedtahir@Users-MacBook-Pro core % mw docker mediawiki fresh npm run selenium-test
Going to remove 220871b66bc1c8c74760703afcfb8fb334d8dbbcde20418065262ccf4d2488e2
Removing 220871b66bc1c8c74760703afcfb8fb334d8dbbcde20418065262ccf4d2488e2 ... done
Creating mwcli-mwdd-default_mediawiki-fresh_1 ... done

> selenium-test
> wdio ./tests/selenium/wdio.conf.js


Execution of 5 spec files started at 2022-01-13T21:33:19.072Z

[0-0] RUNNING in chrome - /tests/selenium/specs/page.js
[0-0] PASSED in chrome - /tests/selenium/specs/page.js
[0-1] RUNNING in chrome - /tests/selenium/specs/recentchanges.js
[0-1] PASSED in chrome - /tests/selenium/specs/recentchanges.js
[0-2] RUNNING in chrome - /tests/selenium/specs/user.js
[0-2] PASSED in chrome - /tests/selenium/specs/user.js
[0-3] RUNNING in chrome - /tests/selenium/specs/watchlist.js
[0-3] PASSED in chrome - /tests/selenium/specs/watchlist.js
[0-4] RUNNING in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js
[0-4] PASSED in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js

 "dot" Reporter:
..............

Spec Files:      5 passed, 5 total (100% completed) in 00:02:52
zeljkofilipin moved this task from Waiting 🏹 to Deep work 🌊 on the User-zeljkofilipin board.Jan 14 2022, 4:17 PM
zeljkofilipin added a comment.Jan 14 2022, 5:22 PM
In T298882#7620022, @Addshore wrote:

Please try mw docker mediawiki fresh npm run selenium-test

The docs for the fresh command are

Examples:
        # Start an interactive terminal in the fresh container
        fresh bash

        # Run npm ci in the currently directory (if within mediawiki)
        fresh npm ci                                    # Run npm ci in the current directory (if within mediawiki)

        # Run mediawiki core tests (when in the mediawiki core directory)
        fresh npm run selenium-test

        # Run a single Wikibase extension test spec (when in the Wikibase extension directory)
        fresh npm run selenium-test:repo -- -- --spec repo/tests/selenium/specs/item.js

This is great! I didn't know about this.

Addshore added a comment.Jan 14 2022, 7:30 PM
In T298882#7622732, @zeljkofilipin wrote:

This is great! I didn't know about this.

You can also / should also be able to run the tests using the quibble images

BUT there are some gotchas there...

Runs commands in a 'quibble' container.

        https://doc.wikimedia.org/quibble/

        THis integration is WORK IN PROGRESS

Usage:
  mw docker mediawiki quibble ... [flags]

Examples:
        # Start an interactive terminal in the quibble container
        quibble bash

        # Get help for the quibble CLI tool
        quibble quibble -- --help

        # Run php-unit quibble stage using your mwdd LocalSettings.php, skipping anything that alters your installation
        quibble quibble -- --skip-zuul --skip-deps --skip-install --db-is-external --run phpunit-unit

        # Run composer phpunit:unit inside the quibble container
        quibble quibble -- --skip-zuul --skip-deps --skip-install --db-is-external --command "composer phpunit:unit"

    Gotchas:
        - This is a WORK IN PROGRESS integration, so don't expect all quibble features to work.
        - quibble will run tests for ALL checked out extensions by default.
        - If you let quibble touch your setup (missing --skip-install for example) it might break your environment.
                - quibble has various things hardcoded :(, for example the user and password for browser tests, you might find the below command helpful.

        mw docker mediawiki exec php maintenance/CeateAndPromote.php -- --sysop WikiAdmin testwikijenkinspass
Addshore moved this task from Backlog to In Progress on the mwcli board.Jan 14 2022, 7:30 PM
Addshore added a comment.Jan 15 2022, 4:52 PM

You can now also find the fresh docs on mw.org https://www.mediawiki.org/wiki/Cli/ref/mw_docker_mediawiki_fresh (automatically generated)

Addshore added a comment.Jan 24 2022, 1:08 PM

@zeljkofilipin do you think this could be closed?

zeljkofilipin assigned this task to Osamaahmed17.Jan 24 2022, 2:26 PM

@Addshore I think so, but let me check.

@Osamaahmed17 does running tests using mwcli work for you on a mac?

Osamaahmed17 added a comment.Jan 24 2022, 2:46 PM

Yes. It worked.

zeljkofilipin closed this task as Resolved.Jan 24 2022, 5:25 PM

@Addshore thanks for your help!

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits