Page MenuHomePhabricator

Wikidata API occasionally returns HTML instead of JSON
Open, Needs TriagePublic

Description

Context: This is C++ code running on VM "wdc" in Toolforge Cloud project "wdq-mm", using the CURL library.

I am querying the Wikidata API from that code, example URL: https://www.wikidata.org/w/api.php?format=json&action=query&meta=siteinfo&siprop=general

Mostly, it works. Sometimes, it returns HTML instead of the specified JSON, more specifically, the API manual, as though I had not used any parameters. Repeatedly querying that URL, even with adding random parameters, results in the same HTML. This will be true for a minute or so, suggesting it's a specific server/cache that the load balancer gives me, then forgets about me.

The last line of the returned HTML seems to be the most helpful for debugging:

<script>(window.RLQ=window.RLQ||[]).push(function(){mw.config.set({"wgBackendResponseTime":242,"wgHostname":"mw1290"});});</script>		</body></html>

Event Timeline

Magnus created this task.Mar 14 2018, 4:23 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 14 2018, 4:23 PM
Reedy removed a subscriber: Cloud-Services.

Why did I get an email for this ticket. This doesn’t concern me.

hoo added a subscriber: hoo.Mar 14 2018, 4:47 PM

@Magnus Did you use GET or POST requests?

Is there a specific user agent sent along these requests which I (or someone else) could look out for when trying to find these requests in the logs?

Why did I get an email for this ticket. This doesn’t concern me.

I have been emailed, too, but was not subscriber.

hoo added a comment.Mar 14 2018, 5:18 PM

Why did I get an email for this ticket. This doesn’t concern me.

I have been emailed, too, but was not subscriber.

That's because initially Cloud-Services was subscribed, so all of its members received an email about this.

Sorry about tagging it Cloud-Services, I thought it might be a cloud config issue (stale DNS or somesuch).

@hoo I tried both GET and POST, same thing.

Error appears sporadically running the exact same binary several times (so less likely an issue with my code):

magnus@wdc:~/wdc$ ./tests
[==========] Running 6 tests from 3 test cases.
[----------] Global test environment set-up.
[----------] 1 test from WikibaseAPI
[ RUN      ] WikibaseAPI.site_info
WikibaseAPI::runQuery: On URL https://www.wikidata.org/w/api.php?format=json&action=query&meta=siteinfo&siprop=general
exception: WikibaseAPI::loadJSONfromCURL: Chunk size 0
magnus@wdc:~/wdc$ ./tests
[==========] Running 6 tests from 3 test cases.
[----------] Global test environment set-up.
[----------] 1 test from WikibaseAPI
[ RUN      ] WikibaseAPI.site_info
[       OK ] WikibaseAPI.site_info (158 ms)
[----------] 1 test from WikibaseAPI (158 ms total)

Hm, it seems to have stopped for now. Can't reproduce it anymore. I'll wait to tomorrow to close it though...