API pageview counts for 'Mobile app' are incorrect since switch to mobile-html
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	SNowick_WMF
	Jun 26 2020, 9:15 PM

Description

The latest versions of both Android Build 2.7.50318 (prod 2020-05-18) and iOS Build 6.6.1 (prod 2020-06-05) mobile apps are using mobile-html to serve content and it appears that the backend has not been adjusted to register pageview counts accordingly. We need to modify the pageview definition to catch the mobile-html views.

We are seeing a significant drop in pageviews filtered by 'mobile app' when querying the /metrics endpoint of the REST API and through Pageviews Analysis on Toolforge. Our Pageviews Superset shows the drop ~ 2020-05-18 when filtered by 'mobile app' as well.

Want to be sure our analytics are setup correctly and that Analytics Engineering is aware of the change. The definition will be updated on the apps, but the webrequests from 2020-05-18 will need to be re-tagged.

@mpopov: so the good news is that webrequests (the source data from which pageview counts we use everywhere are derived) is retained for 90 days, so we can still work with AE to modify the pageview definition (since we can't modify the logged requests) and re-count the pageviews using the updated definition so it catches those mobile-html requests. the bad news is this is extra stress on an already underresourced team, so we'll have to be very nice and understanding

The mobile apps will also need to make changes on their end: T256507 T257389

Other helpful information:
@JoeWalsh: Is the existing PageView counting based on requests to /page/mobile-sections-lead/ for Android and the MobileView API for iOS?
@mpopov : app pageviews are counted by looking at webrequests to the API, see PageviewDefinition.java#L111-L175 and it's looking for /page/mobile-sections/ in the absence of pageview=1 in the X-Analytics header. So as long as the apps' UserAgent validates and the apps set pageview=1 in the mobile-html requests (which are made to the MW API, right?) then the app pageview definition does not need to be modified, although it would be a good idea to make a note of the new endpoint
@JoeWalsh: It looks like we don’t set pageview=1 in either app, so it was probably caught by checking for restAppPageURIPath or iosAppPageURIQuery so there’s now an additional restAppPageURIPath that is /page/mobile-html and it’s used by both iOS and Android (previously only Android used /page/mobile-sections )

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved		None	T256508 API pageview counts for 'Mobile app' are incorrect since switch to mobile-html
Declined		kzimmerman	T256516 Re-process webrequests from 2020-05-18 so that page views from latest Wikipedia app releases are counted
Resolved		JAllemandou	T256515 New app pageview definition needs to be deployed
Resolved		mpopov	T256514 PageviewDefinition should detect /api/rest_v1/page/mobile-html requests as pageviews
Resolved		SNowick_WMF	T256804 Identify next steps for dealing with missing mobile app pageview counts
Duplicate		• Nuria	T257276 Create data quality alarm on access-method
Resolved		SNowick_WMF	T257373 Calculate impact of missing mobile app pageviews to high-level metrics
Resolved	BUG REPORT	• JMinor	T257389 User-Agent sent by the iOS app to mobile-html is incorrect
Resolved		• JMinor	T256507 Add `X-Analytics: pageview=1` header to page view requests
Resolved		mforns	T257692 Add data quality alarm for mobile-app data
Resolved		• Nuria	T257860 Update PageviewDefinition to only include /api/rest_v1/page/mobile-html requests with X-Analytics: pageview=1 in pageviews
Resolved		SNowick_WMF	T257864 Investigate using counts from mobile-html API to stand-in/augment unprocessed mobile app pageview counts
Resolved		SNowick_WMF	T258535 Annotate pageview data to alert users that previously included mobile app pageview data is NOT included in refined pageview datasets

Event Timeline

SNowick_WMF created this task.Jun 26 2020, 9:15 PM

Restricted Application added a subscriber: MusikAnimal. · View Herald TranscriptJun 26 2020, 9:15 PM

SNowick_WMF renamed this task from API pageview counts for 'Mobile app' are off since switch to mobile-html to API pageview counts for 'Mobile app' are incorrect since switch to mobile-html .Jun 26 2020, 9:16 PM

cchen subscribed.Jun 26 2020, 9:22 PM

kzimmerman triaged this task as High priority.Jun 26 2020, 9:50 PM

kzimmerman added a project: Analytics.

mpopov closed subtask T256514: PageviewDefinition should detect /api/rest_v1/page/mobile-html requests as pageviews as Resolved.Jun 30 2020, 7:48 PM

kzimmerman added a project: Epic.Jun 30 2020, 7:49 PM

LGoto moved this task from Triage to Epics on the Product-Analytics board.Jun 30 2020, 7:49 PM

so the good news is that webrequests (the source data from which pageview counts we use everywhere are derived) is retained for 90 days, so we can still work with AE to modify the pageview definition (since we can't modify the logged requests) and re-count the pageviews using the updated definition so it catches those mobile-html requests.

The caveat here is that this is only possible if we have computing resources to do so while continuing computing the regular pageview stream, that implies doubling our computing capacity which I am not sure is possible.

The way to avoid this happening in the future is to move to event-based pageviews (MEP can help) for us, which puts 100% of the control in defining and sending pageview events in the hands of mobile teams.

• Mholloway subscribed.Jul 2 2020, 5:43 PM

mpopov closed subtask T256515: New app pageview definition needs to be deployed as Resolved.Jul 2 2020, 8:30 PM

@JoeWalsh let's talk at your covenience about a path to migrate pageviews for the app to be event based

@Nuria sounds good, I'd happy to talk about that. I set up a meeting to discuss.

JoeWalsh updated the task description. (Show Details)Jul 8 2020, 1:02 AM

JoeWalsh added a subtask: T257389: User-Agent sent by the iOS app to mobile-html is incorrect.

• Nuria mentioned this in T257373: Calculate impact of missing mobile app pageviews to high-level metrics.Jul 8 2020, 2:28 PM

JoeWalsh added a subtask: T256507: Add `X-Analytics: pageview=1` header to page view requests.Jul 10 2020, 1:51 PM

• Charlotte closed subtask T256507: Add `X-Analytics: pageview=1` header to page view requests as Resolved.Jul 13 2020, 2:40 PM

• Charlotte reopened subtask T256507: Add `X-Analytics: pageview=1` header to page view requests as Open.

• fdans added a project: Analytics-Kanban.Jul 13 2020, 4:45 PM

• fdans moved this task from Incoming to Data Quality on the Analytics board.