@kaldari - I am informed that this might be a job for your team. We'd like to transform one of our notifications (currently a local notification) to an Echo notification. How does one go about this?

Confirmed with Dmitry that this isn't needed anytime soon.

+@Dbrant for engineering input.

I poked at this and got it emitting Echo notifications to the user from Suggested Edits milestone events, so now comes the hard part of defining exactly how we want the app to interact with it and what our ultimate goals are.

Echo is primarily intended to create notifications to be displayed in the web UI, and possibly also sent over e-mail. Is the intent here to make unlock notifications available to the user in the web UI (and maybe also e-mail) as well as in the app? If that's so, then we'll need to create the various required notification UI elements (notification message headings, bodies, and icons), or move them over from the app, and I can work with @schoenbaechler on that.

If that's not the intent, and this is strictly for the app to consume, can I ask what the motivation is for wanting to use Echo? Echo itself would take some amount of updating to support the concept of notifications that bypass the web UI completely—at present, based on my testing, disabling notifications on web actually disables the API from returning them as well—and unless I'm missing something, the app can create native notifications from info in the existing wikimediaeditortaskscounts module as easily as from the Echo notifications API module (though admittedly it would probably be cleaner from an engineering perspective for all notifications to come from the same source).

@Mholloway - The last bit of your last sentence is probably the leading motivation for using Echo, as I understand it. Obviously @Dbrant can give much more detail from an engineering perspective.

From a product perspective, I do not care where the notification comes from. I care that the user is informed within the app that they have unlocked the feature, and that they are getting milestone notifications that encourage them to keep editing.

It would be interesting from an 'understanding our users' perspective to see whether users interact with these notifications on web and whether that subsequently drives them back to open the app and continue editing using Suggested Edits, but this is by no means a requirement for this version. If anything, from what you say, it seems like something we'd get as part of dealing with an engineering constraint.

Also adding my two cents here, but let’s start with @Mholloway’s statement:

(...) though admittedly it would probably be cleaner from an engineering perspective for all notifications to come from the same source.

I’d say this is also the expectation from a users perspective. Approaching notifications for Wikipedia holistically is the way to go here. After all, we’re working on the same product (Wikipedia) and share goals regardless of devices/platforms. If we can motivate some people by email or on the web to continue editing anywhere, then we win.

“Suggested edits“ is currently only available on Android, but that is likely to change in the future. In that sense, let’s work on including all app related notifications to Echo. Excited to support @Mholloway with UX/UI input to make it possible.

CC: @Charlotte @Dbrant

In T218599#5214364, @schoenbaechler wrote:

Also adding my two cents here, but let’s start with @Mholloway’s statement:

(...) though admittedly it would probably be cleaner from an engineering perspective for all notifications to come from the same source.

I’d say this is also the expectation from a users perspective. Approaching notifications for Wikipedia holistically is the way to go here. After all, we’re working on the same product (Wikipedia) and share goals regardless of devices/platforms. If we can motivate some people by email or on the web to continue editing anywhere, then we win.

Well, there's an assumption that this type of Echo notification would do something to motivate people on email/web, but we don't know that yet. It is worth testing, certainly, and I want to ensure we do that. I certainly agree that users who engage with Wikipedia on multiple platforms expect to see all their notifications in one place - but this is not necessarily a high proportion of our users. (Logged-in is about 20%, and then what % of that 20% also edit Wikipedia on the web, and thus would expect to see notifications there? @mpopov could help answer this.)

The first thing we need is to understand what is feasible for the initial release in the first couple weeks of July. @Mholloway and @Dbrant, can you please give us a sense of that?

Thanks @Charlotte, happy that you see it that way. Adding a general thought about:

I certainly agree that users who engage with Wikipedia on multiple platforms expect to see all their notifications in one place - but this is not necessarily a high proportion of our users.

In order to create a Wikipedia experience that is unified, continuous and platform independent, we need to embrace opportunities like this. In my eyes, the apps are at the forefront of product innovation for Wikipedia and we have a responsibility.

Engagement of editors on multiple platforms might indeed be low, but if we rely primarily on numbers of the current state for a decision, we’re encouraging a siloed experience. Let’s lead by example and create an inclusive experience. Having Android related echo notifications on the web and email might seem as something neglectable at first, but it’s a powerful sign in the right direction.

In T218599#5215852, @schoenbaechler wrote:

Thanks @Charlotte, happy that you see it that way. Adding a general thought about:

I certainly agree that users who engage with Wikipedia on multiple platforms expect to see all their notifications in one place - but this is not necessarily a high proportion of our users.

In order to create a Wikipedia experience that is unified, continuous and platform independent, we need to embrace opportunities like this. In my eyes, the apps are at the forefront of product innovation for Wikipedia and we have a responsibility.

Engagement of editors on multiple platforms might indeed be low, but if we rely primarily on numbers of the current state for a decision, we’re encouraging a siloed experience. Let’s lead by example and create an inclusive experience. Having Android related echo notifications on the web and email might seem as something neglectable at first, but it’s a powerful sign in the right direction.

Yes, I understand, and agree. That being said, we need to keep the scope reasonable for v1. So let's see what @Mholloway and @Dbrant say about v1 scope, and then prioritise getting these notifications across all platforms accordingly.

Echo notifications will be the cornerstone of what we're doing next quarter anyway, as I mentioned on our call yesterday, so it's not too long to wait in any case. :)

The engineering motivation is one of consistency:
When the user reaches an editing milestone, we'd like the user to be notified through "the notification system" of Wikipedia, which happens to be Echo. If that means seeing notifications in the Web UI and email, then that's absolutely desired. Of course the user might not be able to do much with the web notification, but I think that's a fair tradeoff for having our feature properly plugged into the entire ecosystem. And besides, the only way for the user to unlock this notification is by making edits from the app, so the user will surely have the context to understand what the notification means.

OK! That makes sense about consistency from an engineering perspective, and I like the thinking above about a unified Wikipedia experience.

I am confident that we can do this for v1. I'll open a subtask to create design elements.

Basically the only engineering hurdle left on my side is to figure out the best way of dealing with the "waiting period" after an editing target is passed. I'm curious, have you considered doing away with the "waiting period" in light of the low rate of vandalism and poor quality edits we've seen so far?

@Mholloway In theory I'd be up for doing away with the waiting period in order to see what that actually does to the revert rates. What do we think @Johan and @schoenbaechler?

Do we have any numbers for what difference it would make, so far?

I mean, I really do see the point in not having someone be invited an hour after they started editing if the only reason is that their edits haven't been reverted yet, but that seems like it'd be necessary mainly if it's actually a problem, not just theoretically.

From a UX perspective and given the low revert rates, getting rid of the waiting period sounds good to me. As @Charlotte mentions, let’s closely observe the revert rates if we’re deciding to do so.

So I see no reason not to get rid of the waiting period and observe what happens, but just so we're aware of it: from a vandal-fighting perspective, we'd need to keep this in mind long-term and keep track of what's happening here and remember that the waiting period is an option, in case the app becomes more popular (especially for editing) or vandals find it a convenient way to vandalise things.

But "things could be different in the future" is a bad reason to complicate the issue right now.

Cool. So @Mholloway let's kill the waiting period for captions, and we will ask @mpopov to closely monitor the revert rate for us on these.

In T218599#5214583, @Charlotte wrote:

In T218599#5214364, @schoenbaechler wrote:

Also adding my two cents here, but let’s start with @Mholloway’s statement:

(...) though admittedly it would probably be cleaner from an engineering perspective for all notifications to come from the same source.

I’d say this is also the expectation from a users perspective. Approaching notifications for Wikipedia holistically is the way to go here. After all, we’re working on the same product (Wikipedia) and share goals regardless of devices/platforms. If we can motivate some people by email or on the web to continue editing anywhere, then we win.

Well, there's an assumption that this type of Echo notification would do something to motivate people on email/web, but we don't know that yet. It is worth testing, certainly, and I want to ensure we do that. I certainly agree that users who engage with Wikipedia on multiple platforms expect to see all their notifications in one place - but this is not necessarily a high proportion of our users. (Logged-in is about 20%, and then what % of that 20% also edit Wikipedia on the web, and thus would expect to see notifications there? @mpopov could help answer this.)

I looked at a few languages that are in the top 10 by # of Android app edits. For each of these, I first found users who made article OR title description edits with the (Android) app in April 2019, then looked for additional article edits made on mobile web and desktop. So, here is the April 2019 breakdown of how many Android editors edited just on the app vs other platforms too:

I didn't get the stats for English or other languages because our analytics cluster couldn't handle it so we have to make do with these numbers. From this sample sample – again, these 4 are in the top 10 languages by Android edits – looks like 1/3 of Android editors also edit on mobile web and/or desktop. 2/3 (and nearly 3/4 of Russian Android editors) edit in app only, although that doesn't mean that they don't use the mobile web / desktop interfaces for reading or checking notifications or their watchlist.

Query for future reference, which uses ${snapshot} and ${language_code} placeholders:

USE wmf;
WITH android_editors AS (
  SELECT DISTINCT event_user_text AS username
  FROM mediawiki_history
  WHERE event_entity = 'revision'
    AND (
      (wiki_db = 'wikidatawiki' AND INSTR(event_comment, '|${language_code} */') > 0) -- same language as wiki_db
      OR (wiki_db = '${language_code}wiki')
    )
    AND snapshot = '${snapshot}'
    AND SUBSTR(event_timestamp, 1, 7) = '${snapshot}'
    AND ARRAY_CONTAINS(revision_tags, 'android app edit')
    AND NOT revision_is_identity_reverted
    AND NOT event_user_is_anonymous
), editing_stats AS (
  SELECT
    ae.username,
    SUM(IF(wiki_db = 'wikidatawiki' AND ARRAY_CONTAINS(revision_tags, 'android app edit') AND INSTR(event_comment, '|${language_code} */') > 0, 1, 0)) AS n_wd_desc_edits,
    SUM(IF(wiki_db != 'wikidatawiki' AND ARRAY_CONTAINS(revision_tags, 'android app edit'), 1, 0)) AS n_article_edits_android,
    SUM(IF(wiki_db != 'wikidatawiki' AND ARRAY_CONTAINS(revision_tags, 'mobile web edit'), 1, 0)) AS n_article_edits_mobile_web,
    SUM(IF(wiki_db != 'wikidatawiki' AND NOT ARRAY_CONTAINS(revision_tags, 'mobile edit'), 1, 0)) AS n_article_edits_desktop
  FROM android_editors AS ae
  LEFT JOIN mediawiki_history AS mwh
    ON (
      ae.username = mwh.event_user_text
      AND mwh.wiki_db IN('wikidatawiki', '${language_code}wiki')
      AND mwh.snapshot = '${snapshot}'
    )
  WHERE event_entity = 'revision'
    AND SUBSTR(event_timestamp, 1, 7) = '${snapshot}'
    AND NOT revision_is_identity_reverted
    AND page_namespace = 0 -- article/Q-item ns
  GROUP BY ae.username
)
SELECT
  n_wd_desc_edits > 0 AS edited_title_descriptions,
  n_article_edits_android > 0 AS edited_articles_android,
  n_article_edits_mobile_web > 0 AS edited_articles_mobile_web,
  n_article_edits_desktop > 0 AS edited_articles_desktop,
  COUNT(1) AS n_editors
FROM editing_stats
GROUP BY
  n_wd_desc_edits > 0,
  n_article_edits_android > 0,
  n_article_edits_mobile_web > 0,
  n_article_edits_desktop > 0;

Thanks for this @mpopov - good to know these numbers. So we at least have some sort of addressable audience for multi-platform notifications.

Yesss, thanks much @mpopov for chiming in here – very helpful and interesting! And in regards to...

looks like 1/3 of Android editors also edit on mobile web and/or desktop

... this is way more than I imagined!

Interesting numbers indeed.

The delay is set in configuration, so for now I'll update to set the unlock delay for captions to 0 and not remove any supporting code. If dropping the waiting period proves to be a mistake, getting it back is just a config deployment away.

Retaining support for the waiting period means that I'll have to do a schema update on the targets_passed table so we have a place to track whether unlock notifications have been sent. (Without the waiting period, we could simply fire and forget.) That's something of a hurdle, but hopefully a small one since it's a small table and the change is just adding a boolean column.

Change 513645 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[operations/mediawiki-config@master] WikimediaEditorTasks: Drop caption edit counter unlock delay to 0

https://gerrit.wikimedia.org/r/513645

Change 513645 merged by jenkins-bot:
[operations/mediawiki-config@master] WikimediaEditorTasks: Drop caption edit counter unlock delay to 0

https://gerrit.wikimedia.org/r/513645

I think this can be closed because SE features no longer require unlocking — is that correct?

yup. closing