Page MenuHomePhabricator

Build data controller for importing and refreshing notification center content
Closed, ResolvedPublic

Description

This subtask is for developing the underlying notification data and the methods for importing and refreshing that data. There will be nothing to QA against it until we build UI utilizing this data, so this task can move to PM signoff once development is complete.

The API call to fetch data is here: https://en.wikipedia.org/w/api.php?action=query&meta=notifications&notwikis=enwiki&notlimit=max&notprop=count|list&notformat=model

We must make the corresponding call (change subdomain and notwikis param) for each of the user's app languages, plus common others (wikidatawiki & commonswiki).

  • Initial work - new database migration, some code cleanup, removing legacy polling, capturing additional notifications data into managed object, fetching first page.
  • Instantiate fully realized cell view model from RemoteNotification managed object properties.
  • Refactor fetching first page operation into one import operation, and another refresh operation. Importing will need to continue paging until all pages have been exhausted as indicated by continue in response. Refresh operation will need to continue paging until the oldest object returned already exists in the database. Use a background managed object context for saving these notifications. (Partly done in https://github.com/wikimedia/wikipedia-ios/pull/4047 and https://github.com/wikimedia/wikipedia-ios/pull/4048, needs NSNotification posting).
  • Fetch additional unread notifications from non-app languages. First make fetch call with notcrosswikisummary param (see "API note" section below), then fetch unread notifications from each new project from the crosswikisummary object. (TBD) Note we might need to filter these by recognized language types, so we aren't importing notifications from the rarer notwikis projects. ) (Note: we do not need to fetch read history on these types of notifications. Otherwise that would complicate when exactly to trigger an import since any language with unread notifications could bubble up at any time.)
  • If user changes app preferred languages and adds one, detect which ones are new and kick off import process on new languages.
  • (optional) If user changes app preferred languages and deletes one, clean out data in database housekeeper.
  • Prevent import and refresh operations from executing multiple times back to back. Only one import or refresh process should happen at a time. Also guard against the import processing occurring unnecessarily for a project we have already imported. Persist which projects have been imported, and only allow refreshing once they are imported.
  • Allow import operation to continue importing where it left off if the app gets terminated mid-import. This might involve persisting continue identifiers.
  • Bulk delete all notifications from Core Data upon logout, as well as any other persisted data we needed to keep track of like continue IDs and imported boolean flags.
  • (optional) Move import trigger from notification center appearance to login in the app lifecycle.
  • (optional) Add separate ability sync read and unread flags periodically. There's a chance someone could flip a read flag for a notification deep in history that we have already imported, and thus may not touch again in the refresh step. For this step, we could only fetch unread notifications from each project, be sure unread flag is updated in the database. Pull any local unread notification that did not return from this fetch and flip to read. This is a nice-to-have, we've confirmed that history doesn't need to be totally in sync at all times.
Other useful links:

Notifications API documentation: //www.mediawiki.org/wiki/Notifications/API
Original spike findings: https://phabricator.wikimedia.org/T284239

API note

NotCrossWikiSummary call to make to gather additional unread notifications of interest.
https://{primary language subdomain}.wikipedia.org/w/api.php?action=query&meta=notifications&notwikis={primary language wiki}&notlimit=1&notcrosswikisummary=1

The summary object has an ID of -1 and will return an object of foreign wikis that contain unread notifications.

Example summary object:

{
                    "wiki": "enwiki",
                    "id": -1,
                    "type": "foreign",
                    "category": "other",
                    "section": "all",
                    "timestamp": {
                        "utciso8601": "2021-07-21T17:17:38Z",
                        "utcunix": 1626887858,
                        "unix": "1626887858",
                        "utcmw": "20210721171738",
                        "mw": "20210721171738",
                        "date": "21 July"
                    },
                    "agent": {
                        "id": 35904678,
                        "name": "TSevener (WMF)"
                    },
                    "targetpages": [],
                    "*": {
                        "header": "More notifications from 2 other wikis",
                        "compactHeader": "More notifications from 2 other wikis",
                        "body": "MediaWiki and Test Wikipedia",
                        "icon": "global",
                        "links": {
                            "primary": [],
                            "secondary": []
                        },
                        "iconUrl": "/w/extensions/Echo/modules/icons/global-progressive.svg"
                    },
                    "count": 2,
                    "sources": {
                        "mediawikiwiki": {
                            "title": "MediaWiki",
                            "url": "https://www.mediawiki.org/w/api.php",
                            "base": "https://www.mediawiki.org/wiki/$1",
                            "ts": "2021-07-21T17:17:38Z"
                        },
                        "testwiki": {
                            "title": "Test Wikipedia",
                            "url": "https://test.wikipedia.org/w/api.php",
                            "base": "https://test.wikipedia.org/wiki/$1",
                            "ts": "2021-07-19T16:28:52Z"
                        }
                    }
                },

Related Objects

Event Timeline

Tsevener renamed this task from Data controller for importing and refreshing notification center to Build data controller for importing and refreshing notification center.Jul 23 2021, 8:45 PM
Tsevener renamed this task from Build data controller for importing and refreshing notification center to Build data controller for importing and refreshing notification center content.
Tsevener created this task.
Tsevener removed a project: Epic.
LGoto triaged this task as Medium priority.Jul 26 2021, 6:42 PM

Blocked till we confirm exactly what we're calling to the API.

Wikis to fetch for populating this data:

  1. App languages
  2. Any other wikis with unread notifications? (Row 15 from spreadsheet API options)
  3. Commons + Wikidata (confirmed in Planning)

Just waiting on this, will pick back up when iOS 13 deployment increase task is done.

Tsevener updated the task description. (Show Details)
Tsevener updated the task description. (Show Details)

@cmadeo @OTichonova I'm working on the final pieces of this task which involve scooping up unread notifications from other projects that are not app languages, wikidata or commons. The list under the "notwikis" parameter here are all the possible projects we could get notifications from. If it helps I have stripped all the Wikipedia language wikis that the app already recognizes from this list into this text file:

Can you let me know which of these other projects you would like to see recognized by the app? My best first guess is:

{languageCode}wikibooks
{languageCode}wiktionary
{languageCode}wikiquote
{languageCode}wikimedia
{languageCode}wikisource
{languageCode}wikinews
{languageCode}wikiversity
{languageCode}wikivoyage
specieswiki
commonswiki
wikidatawiki

Keep in mind that for each project we decide to support, we need to know what icon/design would show for that project on the Inbox screen and what would show (either project icon, language name, or a different design) for the notifications center cell. Since some of these other projects are also split up by language, I was unsure which designs we would use.

Screen Shot 2021-12-13 at 9.11.00 AM.png (294×443 px, 31 KB)

Screen Shot 2021-12-13 at 9.11.07 AM.png (164×446 px, 13 KB)

This PR covers scooping up unread notifications from side projects and deleting notifications upon logout: https://github.com/wikimedia/wikipedia-ios/pull/4088.

Hi @Tsevener great question.

We'll want to support:
Wiktionary
Wikiquote
Wikibooks
Wikisource
Wikinews
Wikiversity
Wikispecieis
MediaWiki
Wikidata
Wikimedia commons
Wikivoyage

Only catch is that we don't have icons for most of these projects available. I'll take a try at making them, but it's going to take a little while for me to get them done.

Re: language for sub-projects, for now I'd suggest we not worry about separating them visually in the inbox, but they can be listed with separate toggles in the filters / settings views

Wikisource (English)
Wikisource (Español)

JMinor claimed this task.