Apparently it's because the parameter is not included by default, but I can see the dump with https://en.wikipedia.org/w/api.php?action=query&list=abuselog&afllogid=27143539&aflprop=details. I also believe that this is a sensible default, because computing variables can potentially be expensive (especially if someone is requesting 5000 entries). Would that exact link show the details before?

@Daimona sorry! I had this all wrong. I failed to notice my bot does pass in aflprop=details, and indeed details are there when I test this in the browser (I also agree it's a sensible default to omit details, by the way).

Upon further investigation, my issue seems to do with permissions and/or grants. While logged in as MusikBot II, I can see details in the response for private filters in the browser. Server-side however it logs in using bot passwords. The bot is a sysop (abusefilter-view-private, abusefilter-log-private), and the bot password has the viewrestrictedlogs grant. Despite what the docs say, it's as if the bot doesn't have the abusefilter-log-private permission. The response I get is the same as if you view it while logged out.

Any ideas? I don't think I changed anything; The bot task ran fine for ~1 year until it broke some weeks/months ago, I just never noticed until now.

In T257170#6282495, @MusikAnimal wrote:

Upon further investigation, my issue seems to do with permissions and/or grants. While logged in as MusikBot II, I can see details in the response for private filters in the browser. Server-side however it logs in using bot passwords. The bot is a sysop (abusefilter-view-private, abusefilter-log-private), and the bot password has the viewrestrictedlogs grant. Despite what the docs say, it's as if the bot doesn't have the abusefilter-log-private permission. The response I get is the same as if you view it while logged out.

So, for abuselog details, we first check for abusefilter-log-detail. That's a right given to autoconfirmed users, and it's in the basic grant. This is required for all filters, and is checked regardless of abusefilter-view-private (yes, this is perhaps weird). If the filter is public, this condition is not only necessary but also sufficient. If it's private, then we also require abusefilter-log-private, abusefilter-modify, or abusefilter-view-private. Summing up, you can view details if:

abusefilter-log-details && ( !private || abusefilter-log-private || abusefilter-modify || abusefilter-view-private )

In T257170#6282552, @Daimona wrote:

So, for abuselog details, we first check for abusefilter-log-detail. That's a right given to autoconfirmed users, and it's in the basic grant. This is required for all filters, and is checked regardless of abusefilter-view-private (yes, this is perhaps weird). If the filter is public, this condition is not only necessary but also sufficient. If it's private, then we also require abusefilter-log-private, abusefilter-modify, or abusefilter-view-private. Summing up, you can view details if:

abusefilter-log-details && ( !private || abusefilter-log-private || abusefilter-modify || abusefilter-view-private )

Yeah, so it sounds like it should be working for me, then. I am talking about private filters (filter 1001 to be precise). My bot is a sysop and according to Special:ListGrants#viewrestrictedlogs it should have abusefilter-view-private and abusefilter-log-private (and abusefilter-log-detail which is basic).

In T257170#6282883, @MusikAnimal wrote:

I am talking about private filters (filter 1001 to be precise).

Alright, so it works for public filters? That would be a start...

My bot is a sysop and according to Special:ListGrants#viewrestrictedlogs it should have abusefilter-view-private and abusefilter-log-private (and abusefilter-log-detail which is basic).

Indeed, it should work. Do you have any way to figure out the actual rights held by the bot? It could also be useful to try other AF API modules to see if the permission check is working correctly. For instance, you may try using ApiAbuseFilterCheckSyntax: it requires abusefilter-view-private to be used, so whether you get an error or not would help us somehow.

Sorry for the delay.

I have done more debugging and discovered I was wrong; it is returning the details property, only that is missing added_lines (which is what I need) among several other sub-properties. Here's everything I get when using Bot Passwords:

"user_name",
"user_age",
"page_id",
"page_namespace",
"page_title",
"page_prefixedtitle",
"page_age",
"action",
"summary",
"old_content_model",
"new_content_model",
"old_wikitext",
"new_wikitext",
"edit_diff",
"new_size",
"old_size",
"edit_delta",
"new_html",
"timestamp"

I think if it were a permissions issue, the details is just an empty array. In this case I am getting some data, just not the data I need. Again this is just through Bot Passwords; If I load the URI in my browser while logged in as my bot, I see the full response with added_lines, etc.

In T257170#6288518, @Daimona wrote:

Do you have any way to figure out the actual rights held by the bot?

I used the meta=userinfo API and I believe the bot has all the required rights. The full list: "ipblock-exempt", "noratelimit", "bot", "autoconfirmed", "editsemiprotected", "nominornewtalk", "autopatrol", "apihighlimits", "writeapi", "autoreview", "skipcaptcha", "markbotedits", "titleblacklistlog", "abusefilter-log-detail", "abusefilter-view-private", "abusefilter-log-private", "massmessage", "read", "abusefilter-view", "abusefilter-log", "purge", "spamblacklistlog"

It could also be useful to try other AF API modules to see if the permission check is working correctly. For instance, you may try using ApiAbuseFilterCheckSyntax: it requires abusefilter-view-private to be used, so whether you get an error or not would help us somehow.

My little bot framework doesn't have an entry point for action=abusefilterchecksyntax (or any actions other than query, edit, and a few basic others). So I can't test this just yet, but I am going to figure out how and will get back to you.

I just wanted to update you on my findings so far.

In T257170#6323888, @MusikAnimal wrote:

I have done more debugging and discovered I was wrong; it is returning the details property, only that is missing added_lines (which is what I need) among several other sub-properties.

Great! This leaves us with just a handful of possible explanations.

I think if it were a permissions issue, the details is just an empty array.

Correct. Selectively hiding some variables depending on the permissions is something that we could do, but we don't do that currently. I think we also have a couple of places where a variable might be hidden due to perm issues and the like, but only the value is hidden -- the key is always included in the returned array.

If I load the URI in my browser while logged in as my bot, I see the full response with added_lines, etc.

OK, this is very weird, and is not supposed to happen. Could you please provide me with some handy links for the specific row causing this issue? I'd also like to check the DB and see if there's something wrong there (which wouldn't be surprising).

My little bot framework doesn't have an entry point for action=abusefilterchecksyntax (or any actions other than query, edit, and a few basic others). So I can't test this just yet, but I am going to figure out how and will get back to you.

I think this is no longer necessary. I think this isn't permissions-related.

In T257170#6328380, @MusikAnimal wrote:

I finally figured it out! added_lines is an array, but I was treating it like a string.

Hah, makes sense now :)

I don't quite understand how this was working before... hasn't added_lines been an array for *years* (T52107)? The bot suddenly stopped working on May 11, 2020, after working perfectly since around June 2019. I made no code changes at all, and I don't see any activity to explain the breakage in my bash history on Toolforge.

Internally, yes. I changed the format exposed to clients in r421966. I made it back-compat for the EvalExpression module, but forgot to do the same for QueryAbuseLog (my bad :-/).

I'd like to first apologize for creating this misleading task

These things can happen, and there's no need to apologize :)

I hope no other client was broken by the code change...