Page MenuHomePhabricator

1.31 API Not Returning Data to Parser Function When Logged-In as Admin.
Open, Needs TriagePublic

Description

Troubleshooting and changes to API suggest the API may be the source of this issue.

This code returns correct value in MW 1.30

{{#get_web_data: 
url=https://gunretort.xyz/api.php?format=json&action=parse&page=Portal:TagDescriptions&prop=sections
|format=JSON
|data=SecID=index
|filters=line=CivilRights
}}

{{#external_value:SecID}}

The API url in a browser, returns correct json: https://gunretort.xyz/api.php?format=json&action=parse&page=Portal:TagDescriptions&prop=sections

returns:

{"parse":{"title":"Portal:TagDescriptions","pageid":433,"sections":[{"toclevel":1,"level":"2","line":"Terrorism","number":"1","index":"1","fromtitle":"Portal:TagDescriptions","byteoffset":0,"anchor":"Terrorism"},{"toclevel":1,"level":"2","line":"SecondAmendment","number":"2","index":"2","fromtitle":"Portal:TagDescriptions","byteoffset":747,"anchor":"SecondAmendment"},{"toclevel":1,"level":"2","line":"Timeliness","number":"3","index":"3","fromtitle":"Portal:TagDescriptions","byteoffset":1275,"anchor":"Timeliness"},{"toclevel":1,"level":"2","line":"Money","number":"4","index":"4","fromtitle":"Portal:TagDescriptions","byteoffset":2317,"anchor":"Money"},{"toclevel":1,"level":"2","line":"CivilRights","number":"5","index":"5","fromtitle":"Portal:TagDescriptions","byteoffset":3038,"anchor":"CivilRights"},{"toclevel":1,"level":"2","line":"CommonGround","number":"6","index":"6","fromtitle":"Portal:TagDescriptions","byteoffset":3800,"anchor":"CommonGround"}]}}

But now on MW 1.31, the full get_web_data call is returning:

Error: no local variable "SecID" was set.

Same call works, using static json file. Json file contains identical json as returned by API (above).

{{#get_web_data: 
url=https://gunretort.xyz/sections.json
|format=JSON
|data=SecID=index
|filters=line=CivilRights
}}

{{#external_value:SecID}}

It seems that something about the way the API returns data to a parser function has changed.

Event Timeline

Johnywhy created this task.Jul 1 2018, 10:24 PM
Restricted Application added a project: VisualEditor. · View Herald TranscriptJul 1 2018, 10:24 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Johnywhy renamed this task from 1.31 API Not Returning Data to Template? to 1.31 API Not Returning Data to Parser Function?.Jul 1 2018, 10:26 PM
Johnywhy updated the task description. (Show Details)
Johnywhy added a project: ParserFunctions.
Johnywhy updated the task description. (Show Details)
Anomie added a subscriber: Anomie.

This seems unlikely to have anything to do with any of the projects you tagged. More likely is some bug in MediaWiki-extensions-ExternalData, which is what provides that {{#get_web_data:}} parser function, or some bug in your usage of the extension.

Johnywhy added a comment.EditedJul 2 2018, 4:11 PM

@Anomie the code i'm using works on MW 1.30. Only failed after upgrading to 1.31.
Until we really know the precise cause of the problem, i think you cannot guess what the source is. All code involved is equally suspect-- my code, the extension, MW, and php.

It might not be a bug per se, but may be a change to one of the libs. For example, a change to PHP caused an apparent bug with another extension. It would be easy to assume it's the extension's fault, but it's not.

https://phabricator.wikimedia.org/T196412

Even though the cause was a change in PHP, the fix was to change the extension. But it wasn't a "bug" in the extension.

Anomie added a comment.EditedJul 2 2018, 4:41 PM

Your best way forward would be to do debugging on your wiki that's having the problem to actually identify the cause. There's not much point in trying to guess every possible thing that might be causing your problem, and even less point in trying to add tags for every such thing.

Johnywhy added a comment.EditedJul 2 2018, 4:51 PM
This comment has been deleted.

Still no evidence that this has anything to do with the API.

If something in the API output did change, which I doubt, the fix is to update your use of the parser function to adapt.

If there are some specific additional troubleshooting tasks you'd like me to try, please suggest.

If it were happening on a wiki where I had access to the logs and code, I'd check those logs and temporarily add some debugging statements into the code to see exactly what ExternalData is getting back and how it's processing it.

As it is, the only thing I can do is observe that I can't even reproduce the claimed error by parsing that wikitext.

I agree there's no point in guessing, eg "seems unlikely to have anything to do with..." That's guessing. I see you removed the other tags, but that's based on your guess.

  • My inclusion of the API is based on troubleshooting.
  • Your removal of the API is based on guessing.

On the other hand, it's my job to know MediaWiki well enough to make good guesses about probable causes. And in this case, it seems unlikely that there's anything wrong with the API here. Nor do I see any relation to the ParserFunctions extension, the ApiFeatureUsage extension, the TemplateData extension, or VisualEditor.

Johnywhy added a comment.EditedJul 2 2018, 5:08 PM

ok, found the issue. Disabling anonymous reads on the wiki breaks this.
Now i need to determine which specific page is needed. Not sure how to do that.

Anomie closed this task as Invalid.Jul 2 2018, 5:13 PM

I'm glad you found the issue. (:

I'm going to close this task as "Invalid" since the issue was a misconfiguration on your wiki rather than a problem in any MediaWiki code, and no change in any MediaWiki or Wikimedia code was needed to resolve it.

Johnywhy added a comment.EditedJul 2 2018, 5:53 PM

No, it's not invalid. We still don't know the cause.
Disabling anonymous read when logged in as admin shouldn't break anything.

The extension dev has confirmed that the API has indeed changed with 1.31. Extension dev said the API now requires user to be logged in (or something like that). https://www.mediawiki.org/wiki/Extension_talk:External_Data#1.31_API_Issue:_#get_web_data_Suddenly_Failing

However, I get json data from this API, in a browser, even if i'm not logged into the wiki:

https://gunretort.xyz/api.php?format=json&action=parse&page=Portal:TagDescriptions&prop=sections

This problem is happening when user is logged in as admin, then the API may be failing to recognize the user's logged-in state. Therefor, the API is still suspect.

Here's how i'm disabling anon reads:

$wgGroupPermissions['*']['read'] = false;

(A separate question is, how to make read-functions work when user is not logged in? Not everything is a security risk-- this sounds like too much security. It's also breaking my CategoryTree-- displaying a cat-tree isn't a security risk, and should not require user to be logged in)

@Yaron_Koren

Johnywhy reopened this task as Open.Jul 2 2018, 5:53 PM
Johnywhy updated the task description. (Show Details)
Johnywhy added a subscriber: Yaron_Koren.
Johnywhy renamed this task from 1.31 API Not Returning Data to Parser Function? to 1.31 API Not Returning Data to Parser Function When Logged-In as Admin..Jul 2 2018, 6:11 PM

No, it's not invalid. We still don't know the cause.

Yes, you do: you disabled anonymous reads so the query made by the extension doesn't get the expected results.

Disabling anonymous read when logged in as admin shouldn't break anything.

The query being made by the extension, from the server directly rather than from your browser, is not logged in. It doesn't do anything to forward credentials. And it looks like you should already know this based on your posts at https://www.mediawiki.org/wiki/Extension_talk:External_Data#Use_Logged-In_User_Credentials?.

The extension dev has confirmed that the API has indeed changed with 1.31. Extension dev said the API now requires user to be logged in (or something like that). https://www.mediawiki.org/wiki/Extension_talk:External_Data#1.31_API_Issue:_#get_web_data_Suddenly_Failing

The API code didn't change. You changed your configuration to disable reads.

Or perhaps you removed some hack you had applied (and then forgotten about) that bypassed the read-permission check, or that made queries from the local server be seen as logged-in.

Since this problem is happening when user is logged in as admin, then the API may be failing to recognize the user's logged-in state. Therefor, the API is still suspect.

That assertion is based on a mistaken assumption: the fetch being made from the extension is not in fact logged in.

Johnywhy added a comment.EditedJul 2 2018, 7:20 PM

Ok, thx for clarifying.

I believe i use a process explained here, to login to API, correct?

Re that other thread you linked, it should be possible to login to API as a specific user, correct?

I have no hacks to "bypass permission". I'm not calling this fx with js, just wikitext in template. If you're saying it's possible to bypass permissions with wikitext, please tell how.

Anomie added a comment.Jul 2 2018, 7:45 PM

I believe i use a process explained here, to login to API, correct?

That's what you would do from a client program, yes. That's not going to work from wikitext with ExternalData though. And even if it would, the password for that account would be in the wikitext for anyone to see.

I have no hacks to "bypass permission". I'm not calling this fx with js, just wikitext in template. If you're saying it's possible to bypass permissions with wikitext, please tell how.

I meant hacks like locally editing ApiParse.php to implement isReadMode() to return false, so the module no longer checks for read permission. Or configuration hacks in LocalSettings.php to somehow detect when the request is locally coming from ExternalData and set $wgGroupPermissions['*']['read'] to true instead of false. Or a hack to make MediaWiki somehow or other use a specific logged-in user instead of the default anonymous user when you detect the request is coming locally from ExternalData.

Johnywhy added a comment.EditedJul 2 2018, 7:52 PM

That's what you would do from a client program

Does "Client program" include Common.js?

I meant hacks like locally editing ApiParse.php to implement isReadMode() to return false, so the module no longer checks for read permission. Or configuration hacks in LocalSettings.php to somehow detect when the request is locally coming from ExternalData and set $wgGroupPermissions['*']['read'] to true instead of false. Or a hack to make MediaWiki somehow or other use a specific logged-in user instead of the default anonymous user when you detect the request is coming locally from ExternalData.

Very interesting. But not doing any of those things.

editing ApiParse.php to implement isReadMode()

I'm against hacking the MW core, as i want it to be secure and compliant, and not break on upgrades.

in LocalSettings.php to somehow detect when the request is locally coming from ExternalData and set $wgGroupPermissions['*']['read'] to true instead of false

I prefer code that works as intended, and change the code to meet user-needs, rather than hacking around it.

make MediaWiki somehow or other use a specific logged-in user instead of the default anonymous user

i am wondering if that login process can be used to login as a specific user, instead of the default anonymous user, and if so whether you consider that a "hack". Doesn't sound like a hack to me, as no code is altered.