ApiParse does not have a "noimages" parameter.

MobileFrontend does add such a parameter, though.

@Anomie sorry for having overlooked that!

Deprecate the noimages parameter – since we're extending a core API – in favour of the new and improved disableimages parameter.

Mobile-Apps folks (@BGerstle-WMF, @bearND) : do you use the noimages parameter in any of your requests?

@phuedx Yes, the Android app uses that parameter if the user has selected to not show images.

OK. Then part of this task should be ensuring that Mobile-Apps is notified that the noimages parameter has been deprecated.

I found this ticket while doing a bit of grooming on the neglected Mobile-Apps board a week or so ago, and I have to admit I'm a bit confused. The noimagesparameter isn't marked as deprecated in MobileFrontend, and there is no disableimages parameter (though there is apparently a cookie of that name). We've been happily using noimages for the year+ since this ticket was filed.

Is it still valid? Should we stop using noimages?

OK, I think I finally understand what's going on. This was originally a task about naming consistency, but then took a 90-degree turn and became about not using the noimages param. disableImages is a cookie and always has been, and we should use it instead of noimages (presumably to avoid splitting the cache).

For anyone with expertise and opinions on such matters, a friendly debate inspired by this task over how to handle image disabling in the Android app is happening over at https://gerrit.wikimedia.org/r/#/c/293457/.

@Mholloway: This task isn't that noimages is deprecated, it's that someone thinks that it should be deprecated and disableimages created to be used instead for consistency with the core ApiParse parameters.

If a "disableImages" cookie changes how the page is parsed, it needs to split the cache (and then I'd say a cookie isn't the right way to do it). Otherwise visitor A comes along with the cookie set, the lacking-images response is cached, then visitor B comes along without the cookie but gets the cached output that lacks images anyway. Or vice versa.

@Anomie Oh, I was under the impression from T120151 that we (now) knew how to vary the cache based on disableImages.

(I'm sure my understanding of cache variation is incomplete at best, but I think this means multiple copies of a given page that vary on specific criteria are cached to be served from the same URL.)

Yes, it appears from T120151 that the cache is varied on that cookie; I was mainly responding to your "presumably to avoid splitting the cache" comment there, which is the exact opposite of what T120151 was about.

OK, I think I understand.

In any case, there doesn't appear to be anything else for us to do here other than note that deprecating noimages is under discussion; noted. (For the record, the iOS app doesn't use it.)

Change 402130 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] Remove noimages flag from MobileView API

https://gerrit.wikimedia.org/r/402130

Change 402130 abandoned by Jdlrobson:
Remove noimages flag from MobileView API

https://gerrit.wikimedia.org/r/402130

Change 402130 restored by Jdlrobson:
Remove noimages flag from MobileView API

https://gerrit.wikimedia.org/r/402130

@JoeWalsh what would it take to get rid of the noimages parameter (and eventually the entire mobileview api?)
https://gerrit.wikimedia.org/r/402130 does the removal, but I'm guessing we need to wean certain clients off this API, first?

As far as I know, the noimages parameter is not currently in use by any WMF clients. The Android app stopped using it about a year ago (https://gerrit.wikimedia.org/r/#/c/apps/android/wikipedia/+/389898/), the iOS app never used it (AFAIK), and we're not using it in our lingering mobileview request in MCS.

Thanks @Mholloway I guess this falls on my team then to throw this away. Knowing when we can deprecate the mobileview API (TT186627) from your perspective would also be great to know!

AFAICT there's one client that still uses passes a noimages parameter to the parse API: https://github.com/wikimedia/labs-tools-wikidipendenza

In T99009#4744449, @Jdlrobson wrote:

@JoeWalsh what would it take to get rid of the noimages parameter (and eventually the entire mobileview api?)
https://gerrit.wikimedia.org/r/402130 does the removal, but I'm guessing we need to wean certain clients off this API, first?

Before you can wean clients off the API (if that's even necessary), you need to know who/what they are. Since API requests are web requests, usages of the parse API or the mobileview API with the noimages parameter set will be in the wmf.webrequest table. My recommendation would be to look there first.

Traffic to the api with noimages flag for single day:

select count(*) from webrequest where year = 2018 and month = 11 and day = 1 and uri_query LIKE '%noimages%' AND uri_path LIKE '%api.php%' AND uri_query LIKE "%action=mobileview%";

208, 410

How many of those were unique ip?

select count(distinct client_ip) from webrequest where year = 2018 and month = 11 and day = 1 and uri_query LIKE '%noimages%' AND uri_path LIKE '%api.php%' AND uri_query LIKE "%action=mobileview%";

It turns out a much smaller amount:
37,881

I took a look at user agents:

select user_agent, count(*) from webrequest where year = 2018 and month = 11 and day = 1 and uri_query LIKE '%noimages%' AND uri_path LIKE '%api.php%' AND uri_query LIKE "%action=mobileview%" group by user_agent;

The majority were from Wikipedia app user agent e.g. WikipediaApp v2.*.
Other user agents: Dalvik, http://gottcode.org/connectagram/ and okhttp (the biggest consumer - I'm guessing bots?)

When I cut out the app traffic it becomes clear Apps are not the biggest consumer. In fact the biggest consumer is okhttp (An HTTP & HTTP/2 client for Android and Java applications). Is this also coming indirectly from our apps?

Minus WikipediaApp traffic:

select count(*) from webrequest where year = 2018 and month = 11 and day = 1 and uri_query LIKE '%noimages%' AND uri_path LIKE '%api.php%' AND uri_query LIKE "%action=mobileview%" AND user_agent NOT LIKE 'WikipediaApp/%';

204, 199

Removing the noimages param will send a warning to these clients that we've dropped it and they will still receive content - just they will now be given images where previously they were not (and it will be up to them to remove them). Given the low usage, it feels like we should just cut this off.

Agreed, @phuedx @Mholloway @JoeWalsh ?

If not, the alternative is to mark this a deprecated and revisit this in a few months.

In T99009#4754710, @Jdlrobson wrote:

When I cut out the app traffic it becomes clear Apps are not the biggest consumer. In fact the biggest consumer is okhttp (An HTTP & HTTP/2 client for Android and Java applications). Is this also coming indirectly from our apps?

Our Android app does use okhttp as its HTTP client library, but it's very (deservedly) popular, so it's not clear to me that the requests with 'okhttp' user agents are coming from us. I'm not sure of any case in which that would be sent and not our expected WikipediaApp UA. And those numbers sound quite high at least for our app, considering that in the past @Dbrant looked and found that only a tiny fraction of users had actually chosen to disable images.

Removing the noimages param will send a warning to these clients that we've dropped it and they will still receive content - just they will now be given images where previously they were not (and it will be up to them to remove them). Given the low usage, it feels like we should just cut this off.

Agreed, @phuedx @Mholloway @JoeWalsh ?

I'd better defer to @Dbrant for the official word as regards the Android app. It's a bit touchy because there are certain users who are not able to update their app version because their devices don't run a sufficiently current Android version, and they'll suddenly start receiving images despite not wanting to (presumably to save on data). At the same time, we can't run legacy code forever to support a tiny number of very outdated Android devices, and we have to draw the line somewhere.

Thanks, this is definitely safe to remove from our end. We removed our reliance on the noimages parameter over a year ago, and even then its usage was minuscule. Any remaining users of app versions older than 1 year should update to the latest, or update their OS to accept newer versions.

Okay I've setup T210808 and have created https://www.mediawiki.org/wiki/Extension:MobileFrontend/MobileViewAPI to start plotting a road map here for making this happen. This is stalled until T210808 has happened.

In T99009#4787631, @Jdlrobson wrote:

This is stalled until T210808 has happened.

Is this correct? We've confirmed that the mobile apps don't rely on the noimages parameter, not that they don't rely on the mobileview API.

action=mobileview is still used by the apps, and by MCS (for the main pages).

Here are my notes from Monday's Engineering Managers meeting:

• mobileview
  • • https://phabricator.wikimedia.org/T210808
    • iOS actively uses mobileview but they’re aggressively moving away from it and is moving to the mobile-html endpoint.
    • Android doesn’t use mobileview but is also moving to the mobile-html endpoint.
    • We’re confident that we can mark it as deprecated but we can’t remove it until iOS has moved to mobile-html.
    • Numbers: https://phabricator.wikimedia.org/P7873
  • noimages parameter:
    • https://phabricator.wikimedia.org/T99009
    • Not in use by iOS or Android
    • Numbers: https://phabricator.wikimedia.org/P7884 and https://phabricator.wikimedia.org/P7888
  • iOS & Android apps use action=parse&mobileformat[=true] for the edit preview:
    • Numbers: https://phabricator.wikimedia.org/P7867
    • Speak to @Jhernandez about a RESTful API to render mobile-html formatted previews of wikitext: https://phabricator.wikimedia.org/T211026

☝️ It looks like deprecating and removing the noimages parameter is possible now and deprecating the mobileview API query module is possible but removing it is blocked on a lot of work (most of which is currently being worked on).

Yes, noimages can be removed now.

Change 402130 abandoned by Jdlrobson:
Remove noimages flag from MobileView API

Reason:
Not working on this right now.

https://gerrit.wikimedia.org/r/402130