Hi @ovasileva , in the Metrics collection, the metric to answer below question is vague. "Trust in Wikipedia" is not a metric can be measured by quantitative data.

Does exposing readers to talk pages decrease their overall trust in Wikipedia?

We can discuss it in our meeting on Wednesday.

In T294503#7472877, @jwang wrote:

Hi @ovasileva , in the Metrics collection, the metric to answer below question is vague. "Trust in Wikipedia" is not a metric can be measured by quantitative data.

Does exposing readers to talk pages decrease their overall trust in Wikipedia?

We can discuss it in our meeting on Wednesday.

@jwang - this one we were planning to measure through a quicksurvey - will make it clearer in the list!

The current MobileWikiAppTalk schema has metrics for:
logged-in status
namespace
time_spent

We can explore adding a field for page name, will discuss with @Sharvaniharan in our next meeting.

Will talk page be enabled on both mobile web and mobile app, or just mobile web? @ovasileva

@jwang - this is just for the mobile web

Hi @ovasileva, I have updated instrumentation spec based on our discussion. In the tab of new_in_schema, a few details needs PM and R&D's decisions, marked in Note column.

Update from the Editing Team
@MNeisler is reviewing the Metrics collection sheet to determine:

1. What – if any – additional metrics we ought to consider gathering to answer the questions in the task description's Questions we want to answer section in conjunction
2. Whether the instrumentation necessary to report on the Metrics listed within the Metrics collection sheet is currently implemented

Quick update on my review:

The proposed metrics look good. I added any additional thoughts and comments in the Metrics collection sheet but wanted to highlight one potential issue:

Whether the instrumentation necessary to report on the Metrics listed within the Metrics collection sheet is currently implemented

I remembered we have an open ticket (T249944) re all WikiEditor edits tagged as coming from desktop in EditAttemptStep.  This may impact our ability to reliably distinguish any talk page anon Wikieditor edits coming from desktop vs mobile web for metrics calculated using EditAttemptStep or the new talk_page_edit schema, which I believe is currently only the following metric:

"What percentage of new topics that are created by mobile anons on article talk pages get a response from anyone?"

If confirmed this impacts mobile web talk page edits, we might want to consider re-prioritizing T249944.
CC @ppelberg

In T294503#7485705, @MNeisler wrote:

Quick update on my review:

The proposed metrics look good. I added any additional thoughts and comments in the Metrics collection sheet but wanted to highlight one potential issue:

Whether the instrumentation necessary to report on the Metrics listed within the Metrics collection sheet is currently implemented

I remembered we have an open ticket (T249944) re all WikiEditor edits tagged as coming from desktop in EditAttemptStep.  This may impact our ability to reliably distinguish any talk page anon Wikieditor edits coming from desktop vs mobile web for metrics calculated using EditAttemptStep or the new talk_page_edit schema, which I believe is currently only the following metric:

"What percentage of new topics that are created by mobile anons on article talk pages get a response from anyone?"

If confirmed this impacts mobile web talk page edits, we might want to consider re-prioritizing T249944.
CC @ppelberg

@MNeisler - is it possible to distinguish by skin here? Generally when we're talking about the mobile site here, we're really talking about minerva usage on mobile (and similarly the changes will show on minerva in desktop)

That's approximately what the current meaning of mobile/desktop is in the schema -- if you're in the MobileFrontend experience you get tagged as mobile, otherwise you get tagged as desktop.

In T294503#7485705, @MNeisler wrote:

Quick update on my review:

The proposed metrics look good. I added any additional thoughts and comments in the Metrics collection sheet but wanted to highlight one potential issue:

Whether the instrumentation necessary to report on the Metrics listed within the Metrics collection sheet is currently implemented

I remembered we have an open ticket (T249944) re all WikiEditor edits tagged as coming from desktop in EditAttemptStep.  This may impact our ability to reliably distinguish any talk page anon Wikieditor edits coming from desktop vs mobile web for metrics calculated using EditAttemptStep or the new talk_page_edit schema, which I believe is currently only the following metric:

"What percentage of new topics that are created by mobile anons on article talk pages get a response from anyone?"

If confirmed this impacts mobile web talk page edits, we might want to consider re-prioritizing T249944.
CC @ppelberg

Notes from this morning's conversation

1. T249944 does impact mobile web talk page edits in so far as until T249944 is fixed:
   • Edits made by people who are using the desktop site on a mobile device will be logged as having been published from platform = desktop
   • Edits made by people who are using the mobile site on a mobile device with Javascript disabled will be logged as having been published from platform = desktop
2. "1." means that if we proceed without fixing T249944 when answering "What percentage of new topics that are created by mobile anons on article talk pages get a response from anyone?", we will be excluding some number [i] "new topics that are created by mobile anons" from this calculation

Next steps

• @ppelberg to talk with @MNeisler + @DLynch to learn the level of effort required with fixing T249944
• @ppelberg to decide whether fixing T249944 ought to block us answering "What percentage of new topics that are created by mobile anons on article talk pages get a response from anyone?"

i. Note: we do not currently know how many edits we may potentially be excluding because we do not currently have a way of tracking edits made using the desktop site where platform = mobile (T288972)

To elaborate on my previous comment here:

A little work would need to be done for this to support the change to platform. We'd either want to upstream the mobile device detector from mobilefrontend into mediawiki core so it could be used in multiple places, or agree on some other consistent metric for logging this that instrumentation could consistently use -- one might already exist in some other eventlogging usage. We don't have to use the same thing that triggers mobilefrontend.

We'd also need to change various schema so there's an explicit integration for mobilefrontend.

Oh, specifically, we need a definition of platform that can be consistently applied from client- and server-side logging code.

In T294503#7491267, @ppelberg wrote:

Next steps

• @ppelberg to talk with @MNeisler + @DLynch to learn the level of effort required with fixing T249944

@DLynch, @MNeisler, and I talked about this.

T294503#7493119 documents the approach for fixing T249944 David proposed and Megan, and the wider Product Analytics Team, supports.

• @ppelberg to decide whether fixing T249944 ought to block us answering "What percentage of new topics that are created by mobile anons on article talk pages get a response from anyone?"

While the Editing Team will prioritize work on fixing T249944 by way of implementing what David proposed in T294503#7493119 [i], the analysis being planned in this ticket does NOT need to be blocked on T249944 being resolved. [ii]

i. I've updated T249944's task description with the approach David articulated in T294503#7493119
ii. Thinking: we are assuming that exposing the "talk" tab to mobile anons will cause a nominal increase in the number of new topics mobile anons use the desktop site to create considering the amount of awareness and number of steps required to do so: 1) Click "Talk", 2) Bypass the Add discussion button at the top of the page, 3) Scroll to the bottom of the talk page, 4) Click the Desktop link, 5) Click Edit, 6) etc.

Hey is it possible/on your roadmap to decouple the mobile. Specifically we can't see a difference between Mobile Web and the Apps

Hey is it possible/on your roadmap to decouple the mobile. Specifically we can't see a difference between Mobile Web and the Apps

The schema already has an integration available for app -- presumably the apps should be submitting their events using it, and that would let us see the difference between mobile web and apps? (Well, presumably it should be even more granular such that we can see the difference between android and ios, but...)

Following @DLynch's comment - the apps teams could add an android app or ios app flag to edit events made in app to populate the EditAttemptStep integration column, but where would we need to add that to insure it's captured downstream? We would also need to have android app and ios app made allowable by integration if it is an enum data-type (not sure is that matters in the MEP system but it did in Legacy). Adding @Sharvaniharan and @Dbrant as they will be implementing any code we decide to add. I will follow up with the iOS team as well once this is ironed out.

Following up on this - by default, all edits made in the Android app have an android app edit tag, iOS edits have an ios app edit tag - how do we arrange for the EdittAttemptStep to populate the integration column with these tags? This would allow us to definitively separate out edits made on mobile apps. @DLynch can we discuss getting this implemented?

In T294503#7568135, @SNowick_WMF wrote:

@DLynch can we discuss getting this implemented?

hi @SNowick_WMF – yep. In fact, it looks like we have a meeting scheduled for tomorrow, 14-Dec, at 20:30 UTC, titled Editing + Android metrics for our two teams to talk about this.

Hi EditAttemptStep users (Tagging @DLynch, @MNeisler, @jwang, @ppelberg, , @JTannerWMF, @Dbrant, @Sharvaniharan, @nettrom_WMF, @mpopov; please add others who may need this information going forward.)

Please note that if you are using EditAttemptStep to look at platform = 'phone' data, currently both Android app and regular mobile editing events are categorized under platform = 'phone'. You will need to add a filter to exclude integration = 'app-android' in Turnilo, or add
event.integration != 'app-android' to your queries in order to see only non-app mobile editing data.

Here is an example from Turnilo (2022-02 editing data grouped by platform and pagens with an exclusion filter for `integration = 'app-android' ).