User Details
- User Since
- Oct 26 2014, 10:32 AM (612 w, 1 d)
- Availability
- Available
- LDAP User
- Mushroom
- MediaWiki User
- Unknown
Apr 24 2016
I have noticed that if I set the --compressed option in Curl the file does not get truncated. I searched on Gerrit and found a Varnish configuration change from April 11 whose commit message says:
Note we still need to re-test assumptions about do_stream=false adding Content-Length to responses which lacked them in the common case (no gzip/gunzip on the fly on the response side).
Commit here: https://gerrit.wikimedia.org/r/#/c/282716. Related tasks: T128813, T131501, T131761
A similar issue has been reported by Ivan A. Krestinin here: https://www.wikidata.org/wiki/Wikidata:Contact_the_development_team#Corrupted_response_end.
It happens to him with Wikidata API requests, not WQS, but it has oddly similar features:
- Large response (in my case more than 70 MB)
- Works fine in the browser
- Fails through bots/Python/Curl etc.
- Returns truncated results
- Started about the same date (after 10 April)
Here is the SPARQL, for clarity:
SELECT ?qid ?imdb WHERE {?qid wdt:P345 ?imdb. FILTER regex(str(?imdb), "nm|tt")} ORDER BY ?qid