FWIW (maybe not too much), this is a bit reminiscent of T132965 for the Android app, which turned out to be related to the handling of the X-Analytics header in connection with RESTBase and the rollout of Mobile Content Service there.

Can we check that our user agent and headers have not regressed?

If the solution is not obvious this will need to wait for the next release.

Current headers:

GET /w/api.php?action=mobileview&format=json&noheadings=true&page=Homicide%3A%20Life%20on%20the%20Street%20%28season%206%29&prop=sections%7Ctext%7Clastmodified%7Clastmodifiedby%7Clanguagecount%7Cid%7Cprotection%7Ceditable%7Cdisplaytitle%7Cthumb%7Cdescription%7Cimage%7Crevision&sectionprop=toclevel%7Cline%7Canchor%7Clevel%7Cnumber%7Cfromtitle%7Cindex&sections=all&thumbwidth=640 HTTP/1.1
Host: en.wikipedia.org
Accept: */*
Cookie: WMF-Last-Access=24-Oct-2016; GeoIP=US:PA:Philadelphia:39.94:-75.15:v4
User-Agent: WikipediaApp/5.3.0.0 (iOS 10.0; Phone)
Accept-Language: en
Accept-Encoding: gzip
Connection: keep-alive

@Tbayer does that user agent string look ok to you? Looking here, that seems to be ok:
https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-core/src/main/java/org/wikimedia/analytics/refinery/core/PageviewDefinition.java

For context when this occurred before here was the criteria T131824#2184114

Slightly off… I guess its here:

https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-core/src/main/java/org/wikimedia/analytics/refinery/core/Webrequest.java

So I ran the regex in the file against our current user agent and it matched

Here is the regex from the Webrequest.java file:

"Wikipedia(App|/5.0.)"

Here is the user agent from 5.3:

WikipediaApp/5.3.0.0 (iOS 10.0; Phone)

Blocking on verification from @Tbayer

Hi @madhuvishy and @JAllemandou. We had a similar incident in the spring, and it turned out our user agent had changed. This time, we've checked the user agent against the regex and do not see any issues. Any help or pointers greatly appreciated.

This seems potentially unrelated to app version, but rather OS based. Looks like iOS 10 is not being counted by 9 still works?

In T148663#2742964, @JMinor wrote:

...

This seems potentially unrelated to app version, but rather OS based. Looks like iOS 10 is not being counted by 9 still works?

It does look like that - almost no (although not zero) views from iOS 10:

In T148663#2741645, @Fjalapeno wrote:

So I ran the regex in the file against our current user agent and it matched

Here is the regex from the Webrequest.java file:

"Wikipedia(App|/5.0.)"

Here is the user agent from 5.3:

WikipediaApp/5.3.0.0 (iOS 10.0; Phone)

As Josh mentioned, the Analytics Engineering team will be best equipped to check if this header works with the existing pageview instrumentation.

However, didn't the UA contain the string "iPhone" too earlier? The error could be either in the logic that determines if the request is classified as is_pageview, or in the classification as an iOS app view. I think the latter is handled by the standard ua-parser library, which one would assume has already been updated upstream for any changes resulting from iOS 10.

Good catch @Tbayer :
The function matching iOS mobile-apps pageviews are expecting "iPhone" in their user agent, and also "Wikipedia/5.0." (that is definitely wrong, flexibility for new versions should be builtin).
I suggest some more analysis confirming those are the only issues impacting the drop and update the code.

@JMinor: Please be so kind to communicate changes to apps user agents to us before they happen so we do not find ourselves in this situation.

Can @Fjalapeno please submit a CR to the pageview definition with the needed changes? We would like for you guys to own the process of driving those changes rather than having to play catch-up on our end. We cannot guarantee backfilling pageviews is possible in all instances.

ping @JMinor, ping @Tbayer Please have in mind that we are waiting for a CR from IOS team regarding the proper user agents on definition. We cannot gurantee we can backfill pageviews so the sooner we address the issue the sooner pageviews will be corrected.

Please see my comments regarding backfilling when similar UA changes in the app happen earlier this year: https://phabricator.wikimedia.org/T131824

@dr0ptp4kt this seems out of the scope of iOS team engineering development (submitting CR's for analytics code). Is this something the iOS team should own (not the notification of changes, the actual code submissions)?

@JMinor: we can own the code submission but not w/o notification of detailed changes in a timely manner before they are deployed to users, as we really do not want to be in the situation of having to backfill pageviews.

A sample of OS 10 requests that are not consider pageviews, note tablet versus Phone (not iPhone). @Fjalapeno mentioned some headers above but clearly there is
more variety

{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"0","wmf_app_version":"5.2.1.942"} false WikipediaApp/5.2.1.942 (iOS 10.0; Phone)
{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"0","wmf_app_version":"5.2.0.939"} false WikipediaApp/5.2.0.939 (iOS 10.0.2; Tablet)
{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"0","wmf_app_version":"5.2.0.939"} false WikipediaApp/5.2.0.939 (iOS 10.0.2; Phone)
{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"0","wmf_app_version":"5.2.1.942"} false WikipediaApp/5.2.1.942 (iOS 10.0.1; Tablet)
{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"0","wmf_app_version":"5.2.1.942"} false WikipediaApp/5.2.1.942 (iOS 10.0.3; Phone)
{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"1","wmf_app_version":"5.2.1.942"} false WikipediaApp/5.2.1.942 (iOS 10.1; Tablet)
{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"0","wmf_app_version":"5.2.1.942"} false WikipediaApp/5.2.1.942 (iOS 10.0.1; Phone)
{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"0","wmf_app_version":"5.2.1.942"} false WikipediaApp/5.2.1.942 (iOS 10.0.2; Tablet)
{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"1","wmf_app_version":"5.2.1.942"} false WikipediaApp/5.2.1.942 (iOS 10.1; Phone)
{"browser_major":"-","os_family":"iOS","os_major":"10","device_family":"Other","browser_family":"Other","os_minor":"0","wmf_app_version":"5.2.1.942"} false WikipediaApp/5.2.1.942 (iOS 10.0.2; Phone)

@Nuria Sorry I only provided the phone version of the user agent string. As you pointed out there is a "Tablet" variant.

As to the "iPhone" vs "Phone", I am not sure why "iPhone" is being searched for. According to the file history for this code, we have always used Phone/Tablet:

https://github.com/wikimedia/wikipedia-ios/blob/2b4f10c641a253ddf4d5436883fe1761d16187d2/Wikipedia/Code/WikipediaAppUtils.m#L27

@Nuria can you provide an example of a user agent string from the iOS that was processed correctly?

I know you asked us to communicate changes to you, but I actually don't see any changes in the code and the user agent you posted above look as I would expect. (maybe I am missing something?)

Seeing what worked in the past may shed some light on the issue.

Thanks!

Other notes:
As @JAllemandou pointed out, we should not be looking for hard coded version numbers, so:

Wikipedia(App|/5.0.)

Can be changed to:

WikipediaApp

(Though as far as I can see this wouldn't have caused the regex the fail - and if it did, it would have started failing with version 5.1, not 5.2)

iPhone string is currently necessary in the user_agent for webrequest being considered as pageview, and it looks that new user agents contain iOS only.
Analysis of user_agents in webrequest for this specific case over september and october:

• User agents containg (iOS or iPhone) and (Wikipedia/5.0 or WikipediaApp)
• How many of them actually contain iPhone, and how many contain iOS

SELECT
  month,
  day,
 SUM(CASE WHEN (user_agent LIKE '%iPhone%') THEN 1 ELSE 0 END) AS iphone,
 SUM(CASE WHEN (user_agent LIKE '%iOS%') THEN 1 ELSE 0 END) AS iOS
FROM wmf.webrequest
WHERE webrequest_source = 'text'
  AND year = 2016
  AND month IN (9, 10)
  AND (user_agent like '%iOS%' OR user_agent like '%iPhone%')
  AND (user_agent like '%Wikipedia/5.0%' OR user_agent like '%WikipediaApp%')
GROUP BY
  month,
  day
ORDER BY
  month,
  day
LIMIT 1000;

Plotted results:

As to the "iPhone" vs "Phone", I am not sure why "iPhone" is being searched for. According to the file history for this code, we have always used Phone/Tablet:

@Fjalapeno To triple check: are sure this also applies to webviews?

Those UAS thus far I think they look something like: {"browser_major":"-","os_family":"iOS","os_major":"9","device_family":"Generic Smartphone","browser_family":"Mobile Safari UIWebView","os_minor":"1","wmf_app_version":"5.2.1.942"} false WikipediaApp/5.2.1.942 (iPhone OS 9.1; Phone)

In T148663#2764417, @JAllemandou wrote:

iPhone string is currently necessary in the user_agent for webrequest being considered as pageview, and it looks that new user agents contain iOS only.
Analysis of user_agents in webrequest for this specific case over september and october:

Thanks for checking this! Agree that this recent change in that app's user agent is important and should be documented somewhere. It also affects some other metrics, e.g. the new retention metric where the query used so far had been relying on userAgent LIKE '%iPhone%' as well (T126693#2564106 ).

Verified that several requests that were not marked as pageviews before are marked as pageviews now. Still @Fjalapeno having documentation as to what constitutes a valid user agent in iOS would be of help. You can add docs to here: https://meta.wikimedia.org/wiki/Research:Page_view

@JoeWalsh @Fjalapeno can one of y'all update the user agent definition/pattern for the analytics team per @Nuria previous comment?

For the record, the change related to Nuria's comment above: https://gerrit.wikimedia.org/r/#/c/319374

Confirming that the app's registered pageviews appear to have gone back up to normal on or around November 10 (the above change was merged on November 4):