Page MenuHomePhabricator

SE v4 analytics
Closed, ResolvedPublic

Description

We're about to release Suggested Edits v4 - image tagging. We would like to plug SE v4 edits into the already existing report.

Additionally, I'd like to know:

  • Median number of image tags (+ mode and range) added per image

The beta is out in the store now, so have a play with it.

Event Timeline

Charlotte raised the priority of this task from Medium to High.
Charlotte updated the task description. (Show Details)

@Dbrant how are these events tagged in the db - is there a different indicator for a Suggested tag and a manually entered one? Is the event going to mobilewikiappsuggestededits and mobilewikiappedit tables?

@SNowick_WMF - The suggested vs manually entered tag thing will require a new schema. When published, the event goes to mobilewikiappsuggestededits but NOT mobilewikiappedit.

@SNowick_WMF - The suggested vs manually entered tag thing will require a new schema. When published, the event goes to mobilewikiappsuggestededits but NOT mobilewikiappedit.

Ok, we can plan for that but not sure if we need a new schema, we can also add new event/action tags to the SuggestedEdits table that tracks other editing events, @Dbrant has said he will discuss with you in next meeting.

Also note @Charlotte the image tag event is written to both tables, just want to update this for future reference.

Good news, cheers @SNowick_WMF. @Sharvaniharan also found a bug in the tracking that I'll ask her to explain more thoroughly, since it might throw us off a bit.

@SNowick_WMF we'll sync up to go over the new additions to the analytics and ways to best make use of them.

The Google Vision tagging is not active, once it is enabled we can measure usage vs manually added tags. Is there is an alternate metric we want added to the Suggested Edits report in the meantime?

We're not using Google Vision - we took it out of the release. So it should just be the manually added tags you see in the report, sorry. In addition to adding the image tags to all the existing graphs, I'd like to see:

Median number of tags (+ mode and range) added per image

SNowick_WMF added a subscriber: Sharvaniharan.

Data in mobilewikiappedits source: #suggestededit-imgtag-custom 1.0 action: saved Feature live date: 2020-05-18
Source tags:
#suggestededit-imgtag-auto 1.0 #suggestededit-imgtag-custom 1.0

Example: https://commons.wikimedia.org/wiki/Special:Contributions/Dbrant_testing

Suggested Edits automated report has been updated where data was available, screenshots attached. I will be running a manual update to the report and submitting until the rest of the report is running and uploaded online.

The report also requires data from MobileWikiAppSuggestedEdits table, which needed a fix so data is formatted as expected in the revision of the schema. @Dbrant has implemented fix in Beta and I will validate the data when it is propagated.

Screen Shot 2020-06-04 at 9.03.25 AM.png (882×2 px, 580 KB)

Screen Shot 2020-06-04 at 9.04.19 AM.png (662×1 px, 239 KB)

Screen Shot 2020-06-04 at 9.04.49 AM.png (1×1 px, 336 KB)

Great stuff - thanks @SNowick_WMF.

cc @JKatzWMF who will be interested in the stats.

Updated graphs as of 2020-06-08 here: Image Tags

Charlotte lowered the priority of this task from High to Medium.Jun 9 2020, 6:45 PM
Charlotte raised the priority of this task from Medium to High.Jun 9 2020, 6:51 PM

As discussed we are changing the reporting dimension, where possible, from the previously measured Inside vs Outside Suggested Edits (since all edits are now considered 'inside') to showing the source from where the edit action originated. I'm going to save the V2 report and move to v4 as our working automated report when it is complete.

Need to specify which edits now have CTA prompts:
Image captions sourced from Suggested Edits, gallery and articles (As reported here)

@Charlotte are there other edits that have CTAs we want to track currently? As we add CTAs we can add reporting for those sources.

@SNowick_WMF - Article descriptions also have CTA prompts, from the article page. When T254142 is done, we will have image tag prompts from the Commons filepage for images.

Basic metrics for image tags now live on Superset https://superset.wikimedia.org/superset/dashboard/androidimagetags/

More extensive reporting on CTA prompt success in progress.

Counts for Image Tags have been added to Android Monthly report