Page MenuHomePhabricator
Create Task
Maniphest T362723

Data Validation for iOS Image Recs
Open, HighPublic
Actions

Description

Once Image Recommendations is in Test Flight and available to beta testers, test and validate instrumentation.

Details

SubjectRepoBranchLines +/-
Look for iPadOS in user-agent, in addition to iOS.mediawiki/extensions/MobileAppwmf/1.43.0-wmf.9+1 -1
Look for iPadOS in user-agent, in addition to iOS.mediawiki/extensions/MobileAppmaster+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

HNordeenWMF created this task.Apr 16 2024, 9:43 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 16 2024, 9:43 PM
HNordeenWMF triaged this task as Low priority.Apr 23 2024, 6:11 PM
HNordeenWMF moved this task from Needs Triage to Up next on the Wikipedia-iOS-App-Backlog board.
HNordeenWMF updated the task description. (Show Details)Apr 24 2024, 6:45 PM
Tsevener mentioned this in T359131: [M] Create schema docs and instrumentation plan for iOS Image Recs.Apr 26 2024, 2:36 PM
Tsevener edited projects, added Wikipedia-iOS-App-Backlog (iOS Release FY2023-24); removed Wikipedia-iOS-App-Backlog.
Tsevener moved this task from Tasks from Product Backlog to Needs Analytics on the Wikipedia-iOS-App-Backlog (iOS Release FY2023-24) board.
Tsevener subscribed.

Build is TestFlight 7.5.0 (3509) for beta testing validation.

Tsevener added a comment.Apr 30 2024, 8:07 PM

Other external TestFlight beta builds: TestFlight 7.5.0 (3525) and TestFlight 7.5.0 (3511).

Tsevener raised the priority of this task from Low to High.Apr 30 2024, 8:10 PM
SNowick_WMF added a comment.EditedMay 1 2024, 7:40 PM

Validating we are seeing almost all events, full action_data for successful edits and edit events tracked in editattemptstep

Data Validation

I am not seeing the following events yet:

actionactive_interface

|view_edit_help| editsummary_dialog
|view_watchlist_help| editsummary_dialog
|impression|nosuggestions_dialog (I am seeing nosuggestions_back action, so this should be firing too)

GitHubPRBot added a comment.May 1 2024, 9:13 PM

tonisevener opened https://github.com/wikimedia/wikipedia-ios/pull/4832

Tsevener added a comment.May 1 2024, 9:14 PM

@SNowick_WMF Thanks! I made a PR to fix the missing impression event. I was able to trigger view_edit_help and view_watchlist_help, so maybe beta testers aren't tapping those buttons?

Tsevener claimed this task.May 1 2024, 9:46 PM
Tsevener moved this task from Needs Analytics to Needs Code Review on the Wikipedia-iOS-App-Backlog (iOS Release FY2023-24) board.
GitHubPRBot added a comment.May 2 2024, 9:56 AM

mazevedofs closed https://github.com/wikimedia/wikipedia-ios/pull/4832

Mazevedo moved this task from Needs Code Review to Needs Analytics on the Wikipedia-iOS-App-Backlog (iOS Release FY2023-24) board.May 2 2024, 1:47 PM
Mazevedo subscribed.

PR with fixes merged, we'll validate it in the release

Tsevener removed Tsevener as the assignee of this task.May 2 2024, 2:54 PM
HNordeenWMF added a comment.May 3 2024, 3:28 PM

Hi @SNowick_WMF , could you please plan to validate that Watchlist Toggle instrumentation change as well in this task? https://github.com/wikimedia/wikipedia-ios/pull/4840
Thanks!

HNordeenWMF mentioned this in T354219: [L] Improve Edit Summary on iOS .May 3 2024, 3:29 PM
SNowick_WMF added a comment.May 6 2024, 8:27 PM

Validating we are seeing prior missing events from above:

actionactive_interface
view_edit_helpeditsummary_dialog
view_watchlist_helpeditsummary_dialog
impressionnosuggestions_dialog
JTannerWMF moved this task from Needs Analytics to Ready for PM Signoff on the Wikipedia-iOS-App-Backlog (iOS Release FY2023-24) board.May 13 2024, 2:55 PM
HNordeenWMF moved this task from Ready for PM Signoff to Ready for Release on the Wikipedia-iOS-App-Backlog (iOS Release FY2023-24) board.Thu, May 23, 9:31 PM
SNowick_WMF added a comment.Wed, Jun 5, 12:36 AM

@Tsevener I am re-opening this so we can investigate an issue I found in edit data while doing 15 Day Image Rec analysis - we can discuss in App Data meeting.

Details:
While querying for Image Rec edits I ran into 74 edits where we had revision_ids in our android_image_recommendation_event data that I couldn't match in data from my regular mediawiki_historyquery that looks for ios app edit in the revision_tags column.

I pulled the edit data by the revision_id from android_image_recommendation_event and found that for all of these edits there was no ios app edit value in the revision_tags - the column only showed ['mobile edit', 'mobile app edit'] - these were known iOS app edits (with Added image, etc. in event_comment for most of them) but for some reason they weren't properly attributed as ios app edit- this happened across multiple wikis:

wikirev_id (Count Unique)
enwiki63
dewiki5
cawiki2
frwiki2
itwiki1
trwiki1
gerritbot added a comment.Wed, Jun 5, 9:23 PM

Change #1039302 had a related patch set uploaded (by Dbrant; author: Dbrant):

[mediawiki/extensions/MobileApp@master] Look for iPadOS in user-agent, in addition to iOS.

https://gerrit.wikimedia.org/r/1039302

gerritbot added a project: Patch-For-Review.Wed, Jun 5, 9:23 PM
SNowick_WMF mentioned this in T366756: Add iOS to WikipediaApp value in useragent for all in app events.Fri, Jun 7, 5:10 PM
Tsevener added a comment.Wed, Jun 12, 9:24 PM

Patch looks good, I tested with both iPhone and iPad user agents. I tried to test a Vision Pro user agent, but I could not get my Vision Pro simulator to appear for testing.

gerritbot added a comment.Wed, Jun 12, 9:48 PM

Change #1039302 merged by jenkins-bot:

[mediawiki/extensions/MobileApp@master] Look for iPadOS in user-agent, in addition to iOS.

https://gerrit.wikimedia.org/r/1039302

ReleaseTaggerBot added a project: MW-1.43-notes (1.43.0-wmf.10; 2024-06-18).Wed, Jun 12, 10:00 PM
gerritbot added a comment.Thu, Jun 13, 2:52 PM

Change #1043110 had a related patch set uploaded (by Dbrant; author: Dbrant):

[mediawiki/extensions/MobileApp@wmf/1.43.0-wmf.9] Look for iPadOS in user-agent, in addition to iOS.

https://gerrit.wikimedia.org/r/1043110

gerritbot added a comment.Thu, Jun 13, 8:42 PM

Change #1043110 merged by jenkins-bot:

[mediawiki/extensions/MobileApp@wmf/1.43.0-wmf.9] Look for iPadOS in user-agent, in addition to iOS.

https://gerrit.wikimedia.org/r/1043110

ReleaseTaggerBot edited projects, added MW-1.43-notes (1.43.0-wmf.9; 2024-06-11); removed MW-1.43-notes (1.43.0-wmf.10; 2024-06-18).Thu, Jun 13, 9:00 PM
Stashbot added a comment.Thu, Jun 13, 9:01 PM

Mentioned in SAL (#wikimedia-operations) [2024-06-13T21:01:43Z] <jsn@deploy1002> Started scap: Backport for [[gerrit:1043110|Look for iPadOS in user-agent, in addition to iOS. (T362723)]]

Stashbot added a comment.Thu, Jun 13, 9:04 PM

Mentioned in SAL (#wikimedia-operations) [2024-06-13T21:04:09Z] <jsn@deploy1002> dbrant, jsn: Backport for [[gerrit:1043110|Look for iPadOS in user-agent, in addition to iOS. (T362723)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Stashbot added a comment.Thu, Jun 13, 9:15 PM

Mentioned in SAL (#wikimedia-operations) [2024-06-13T21:15:54Z] <jsn@deploy1002> Finished scap: Backport for [[gerrit:1043110|Look for iPadOS in user-agent, in addition to iOS. (T362723)]] (duration: 14m 11s)

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL