libcurl 7.85+ breaks DNS in mwcli
Open, Needs TriagePublicBUG REPORT
Actions

Assigned To

None

Authored By

	Ollie.Shotton_WMDE
	Mar 10 2025, 1:24 PM

Description

Steps to replicate the issue (include links if applicable):

For the mediawiki service, use a container image that uses libcurl v7.85+
- Update to mwcli v0.27.1 and update the docker environment
  - sudo mw update --version=v0.27.1
  - mw docker update
- Use the bookworm-php83-fpm:1.0.0 image and recreate the mediawiki service
  - mw docker env set MEDIAWIKI_IMAGE docker-registry.wikimedia.org/dev/bookworm-php83-fpm:1.0.0
  - mw docker mediawiki create -- force-recreate
- See the curl/libcurl version: $ mw docker mediawiki exec -- curl --version

Execute a curl command, e.g.

$ mw dev mw exec -- curl --request GET --url 'http://default.mediawiki.mwdd.localhost:8080/w/rest.php/wikibase/v1/entities/items/Q1' --header 'User-Agent: whatever' -v
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 127.0.0.1:8080...
* connect to 127.0.0.1 port 8080 failed: Connection refused
*   Trying [::1]:8080...
* connect to ::1 port 8080 failed: Connection refused
* Failed to connect to default.mediawiki.mwdd.localhost port 8080 after 0 ms: Couldn't connect to server
* Closing connection 0
curl: (7) Failed to connect to default.mediawiki.mwdd.localhost port 8080 after 0 ms: Couldn't connect to server

Run Wikibase REST API e2e tests involving sitelinks (execute from extensions/Wikibase/repo/rest-api dir): $ mw dev mw fresh -- npm run api-testing:e2e-crud -- --grep sitelink

Software version:

$ mw version
Version Information  Value                                                     
BuildDate            2025-03-06T17:08:42Z                                      
Version              0.27.1                                                    
Releases             https://gitlab.wikimedia.org/repos/releng/cli/-/releases

Workaround:

Update to v0.27.2+ or manually set the image the mediawiki service uses to one that include a version of libcurl before 7.85.0:

mw dev env set MEDIAWIKI_IMAGE docker-registry.wikimedia.org/dev/buster-php81-fpm:1.0.1-s2
mw dev mw create --force-recreate

Other information (browser name/version, screenshots, etc.):

PHP uses libcurl so the solution can't be as simple as "don't use curl"
The change was made in PR9296 and released in curl/libcurl 7.85 (see the tags on the commit)
Discussions relating to this change breaking peoples workflows and curl maintainers reluctant to revert the change: #11104 and #15628
More information in T387494#10617603

TODO:

Roll back to using the buster-php81-fpm image by default until a solution implemented
Decide on what to use instead of *.localhost
Implement alternative
Add more test cases to CI to cover similar issues in the future

Details

	Title	Reference	Author	Source Branch	Dest Branch
	Revert to `buster-php81-fpm:1.0.1-s2` by default for mediawiki services	repos/releng/cli!608	ollieshotton	fix-v0.27-release	main

Customize query in GitLab

Related Objects

Mentioned In: T387494: expose sub-commands causes DNS issues
Mentioned Here: T292897: [mwcli dev] investigate and maybe build in support for ngrok or localtunnel
T387494: expose sub-commands causes DNS issues

Event Timeline

Ollie.Shotton_WMDE created this task.Mar 10 2025, 1:24 PM

Ollie.Shotton_WMDE mentioned this in T387494: expose sub-commands causes DNS issues.

Ollie.Shotton_WMDE updated the task description. (Show Details)Mar 10 2025, 1:26 PM

.local can't be used as it is reserved by the IETF for mDNS/zeroconf - https://en.wikipedia.org/wiki/.local
.internal might be a good replacement as it is reserved by ICANN for private application use that may not be installed as a TLD - https://en.wikipedia.org/wiki/.internal

My summary from the other ticket

In T387494#10617603, @Addshore wrote:

This is an issue with the curl CLI tool and libcurl (and maybe others depending on how they read the spec) using *.localhost to resolve to non loopback ips

Having to use different hosts for different tools / inside and outside containers is annoying

Having to configure custom DNS on the host machine of mwcli is also annoying

Would there be much point in switching to something like .internal when it doesn't give us any benefits (when comparing to *.mwdd which is already also used throughout?)

I believe the only changeover required to make this the default would be https://gitlab.wikimedia.org/repos/releng/cli/-/blob/main/internal/mwdd/files/embed/mediawiki/MwddSettings.php?ref_type=heads#L50 and other things in the settings that refer to .localhost as a default.
Of course, changing this breaks bullet point 3 above, and you'd then have to define custom DNS in order to have it work out of the box in a browser.

I always considered having MediaWiki dynamically determine what $wgServer host should be used, but didn;t investigate too much at the time as everything was fine.
If $wgServer could be dynamically set (and wouldnt break anything in MediaWiki etc) then we could probably support both cases throughout.

I imagine this would get messy if services that make part of the development environment would not consistently use one or the other? And this is probably also an issue if services / users use different ones.
For example, if the WDQS updater uses .mwdd, it would write triples with .mwdd in the URI I believe. And a user then visiting the .localhost wiki would be confused possibly?

This dynamic hostname stuff is also roughly related to T292897: [mwcli dev] investigate and maybe build in support for ngrok or localtunnel which in turn would require another set of hostnames to work.

Looking at your example above @Ollie.Shotton_WMDE I'm a little confused as to why your culr request to default.mediawiki.mwdd:8080 resulted to 127.0.0.1:8080 anyway?

This is not a *.localhost so I don't see how it is imediatly related to the curl issue discussed in the other ticket and on the github issues?

Per the DNS server, this should resolve to the nginx instance which would have a different IP https://gitlab.wikimedia.org/repos/releng/cli/-/blob/main/internal/mwdd/files/embed/base.yml?ref_type=heads#L23

$ mw dev mw exec -- curl --request GET --url 'http://default.mediawiki.mwdd:8080/w/rest.php/wikibase/v1/entities/items/Q1' --header 'User-Agent: whatever' -v
Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying 127.0.0.1:8080...
* connect to 127.0.0.1 port 8080 failed: Connection refused
*   Trying [::1]:8080...
* connect to ::1 port 8080 failed: Connection refused
* Failed to connect to default.mediawiki.mwdd.localhost port 8080 after 0 ms: Couldn't connect to server
* Closing connection 0
curl: (7) Failed to connect to default.mediawiki.mwdd.localhost port 8080 after 0 ms: Couldn't connect to server

Ollie.Shotton_WMDE updated the task description. (Show Details)Mar 10 2025, 3:09 PM

Would there be much point in switching to something like .internal when it doesn't give us any benefits (when comparing to *.mwdd which is already also used throughout?)

.internal is reserved rather than just currently unused. There were issues when .dev became a TLD. Granted .mwdd is less likely than .dev to become a TLD, but given there is an alternative that is guaranteed to not become a TLD then I think we should probably use that across the board (and not *.mwdd)

Looking at your example above @Ollie.Shotton_WMDE I'm a little confused as to why your culr request to default.mediawiki.mwdd:8080 resulted to 127.0.0.1:8080 anyway?

Sorry, I copied the wrong command, that was meant to be a *.localhost address - have fixed the task description.

DAlangi_WMF awarded a token.Mar 10 2025, 3:19 PM

Ollie.Shotton_WMDE updated the task description. (Show Details)Mar 11 2025, 10:41 AM

ollieshotton opened https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/608

Revert to buster-php81-fpm:1.0.1-s2 by default for mediawiki services

ollieshotton merged https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/608

Revert to buster-php81-fpm:1.0.1-s2 by default for mediawiki services