Page MenuHomePhabricator

Offline editing to support people with intermittent or no internet access (e.g. Kiwix, mobile)
Open, NormalPublic

Description

Several discussions are the genesis of this ticket. Off line editing will enable people with limited or intermittent access to the internet to contribute to wiki projects more effectively.

Offline reading of Wikimedia projects is a significant and relatively well-supported use case. However, full access requires not just the ability to read but the ability to edit. This distinction is not particularly significant when we consider, for example, a Canadian user who reads Wikipedia offline during a backpacking vacation in South America and retrieves her ability to edit the articles she reads as soon as she flies back home. But it does become significant when we consider, for example, a Congolese refugee who has lived in a Burundian refugee camp since 2012.

This has two major aspects (which might be filed as blocking tasks).

  1. Technical: Offline editing means that a user makes changes to a local copy of a wiki and, at some later date, their client synchronizes the changes to Wikimedia's servers. This is a significant challenge, but would share much of the technical work necessary for T3898.
  2. Community: Provide adequate documentation targeted specifically to Global South users and facilitators. Define particular tasks (like translation of global language content into local languages) which these users are well suited to undertake. Find projects which are interested in receiving such contributions and editors who are interested in helping with the process.

Details of the use case

Offline: The changes would be synchronized a low-bandwidth, unreliable internet connection (perhaps once a day) or by a sneakernet made up of development staff (perhaps once a month).
Facilitated: This task focuses on the case of a physical community of local users (likely with low digital literacy) supported by a facilitator or teacher who is more digitally literate but probably does not have specific Wikimedia experience.

Other projects to consider

History

This task originated from a request posed by the French development group Bibliothèques Sans Frontières at the Wikimedia Hackathon 2015. BSF has developed the Ideas Box, a portable media center designed as a kit that fits on two pallets and can be installed in less than 20 minutes. The box creates a cultural space covering 330 sq ft and includes a Koombook server running open-source Python software which serves as an autonomous and ultra-portable digital library. The device creates a Wi-Fi hotspot which gives connected devices access to locally stored content from Wikipedia, Khan Academy, TEDtalks, a curated selection from the Gutenberg Library, and thousands of other documents and videos. The KoomBook creates a WiFi hotspot that users can connect to with a laptop, tablet or smartphone. Up to 20 simultaneous users can download or upload content that will automatically update when the KoomBook has access to the internet.

BSF's challenge to Wikipedia users:

Wikipedia offline is one of the most popular resources in the Ideas Box and Koombook. But it risks becoming yet another source of content created in the Global North and dumped in the Global South. We are working with Wikimedia on training our users on contributing to Wikipedia. But that currently cannot be done in many of our projects as internet is highly unreliable and/or extremely expensive. And these are some of the richest context in terms of uniqueness of contribution (language, topics etc.). Can we come up together with a solution for asynchronous contribution for low connectivity contexts?

Use cases

Bibliothèques Sans Frontières

The French development group Bibliothèques Sans Frontières has created the Ideas Box, a portable media center which, among other things, offers access to offline educational materials including Wikipedia over Wi-Fi. Up to 20 simultaneous users can upload or download content that will automatically update when the server has access to the internet.

BSF has installed Ideas Boxes in places like refugee camps in Burundi, many of which have only sporadic, low-quality internet access or none at all. Sites with sporadic access might be able to synchronize once a day; sites with none would only be able to synchronize by sneakernet, perhaps once a month.

BSF considers it an ethical imperative to offer Wikipedia editing alongside Wikipedia reading: "Wikipedia offline is one of the most popular resources in the Ideas Box...But it risks becoming yet another source of content created in the Global North and dumped in the Global South."

For more information about BSF's use case, you can contact Barbara Schack, director of development, at barbara.schack@bibliosansfrontieres.org.

See how an article looks like with offline access (no edit button at all):

See also

Event Timeline

aripstra created this task.Jul 24 2015, 9:45 PM
aripstra updated the task description. (Show Details)
aripstra raised the priority of this task from to Needs Triage.
aripstra claimed this task.
aripstra added subscribers: aripstra, awight.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 24 2015, 9:45 PM

Here is some good info on the topic -

Small thing I wanted to talk to editors about, "friendly undo":
name

Here's the longish and crazyish overview of what else could be done:
name

This version touches a bit more on offline editing:
name

aripstra updated the task description. (Show Details)Jul 30 2015, 6:41 PM
aripstra added a project: Mobile.
aripstra set Security to None.
Qgil updated the task description. (Show Details)Sep 23 2015, 8:09 AM

There are several tasks related to this use case, and maybe a first step in order to get attention / resources would be to converge the discussions in a single task.

Also, what about framing this feature in the context of Kiwix, our popular offline reader? The only way to edit is to have the content available, and regular Wikipedia will not offer such content when users are offline. Maybe editing has already been discussed in the context of the Kiwix project. No #Kiwix in Phabricator, so I could not search. :) @Kelson might know.

Qgil added a comment.Sep 23 2015, 8:43 AM

Also see related discussion in https://lists.wikimedia.org/pipermail/wikitech-l/2015-September/083335.html which links to related ideas / URLs

A couple things to consider when addressing the intermittent connectivity case:

  • Some users may go a long time without connection and will require a lot of pages to be updated. Some may only go days or even hours without connection, but they require those updates as soon as possible.
  • Also, some users might have high-speed internet when they do connect, but others may still be limited on bandwidth. As such, they would benefit from some sort of triage system, allowing them to update the content they deem most important and getting to lower-priority content if their connection lasts long enough.
cscott added a subscriber: cscott.Sep 23 2015, 5:44 PM

This is an issue we wanted to address for the OLPC project, but never quite finished. http://nell-wikipedia.github.cscott.net/ was the offline app we started to build for this purpose. (Kiwix wasn't quite appropriate for our needs, since it didn't allow incremental update of ZIM bundles, and (at the time at least) didn't allow us to package subsets of wikipedia, with reduced-size images, and other features we needed for our deployments.)

A modern attempt to address this is likely to include some of the same pieces as for T113004: Make it easy to fork, branch, and merge pages (or more) --- in particular, T108664: Provide an interactive edit conflict resolution tool and T26617: Implement diff on sentence level (instead of per paragraph block) and other diff/merge improvements). T112984: Real Time Collaborative Editing will be relevant as well, since the visual-editor-related infrastructure for synchronizing changes among a group of real-time editors turns out to be the same infrastructure needed for synchronizing changes among more-disconnected users.

There is also a significant UX issue, which it would be nice to address, perhaps in the context of the Android app. Most serious offline tools (for example, wwwoffle) have a mechanism to queue pages for later download and visualize and manipulate that queue (for example, cancel pages you are no longer interested in). Edited pages would show up in the queue as well, as items to upload the next time connectivity is established, and perhaps they are prioritized over downloads. If there are conflicts with the edited version, enough information needs to be downloaded to allow the conflict to be resolved interactively offline, since the window of connectivity might be very short.

Further, this "connectivity queue" (requested downloads and uploads of edited content) could ideally be transferred onto some other media for sneakernet. For example, in many places in Peru schools don't have connectivity, but all teachers are required to make a trip into the nearest large city for training and administrative tasks once a month. The desired use case would allow the teacher to collect all the "queues" from the students' devices (phones, tablets, OLPC-type laptops, etc) onto a flash disk or SD card, satisfy the request while in the city, and return to the school to distribute the downloaded content. It would be useful to keep these users in mind when designing the offline UX.

Nominated as a candidate for the wikimedia developer summit.

Adding a response to T100154#1667080 here per request to consolidate discussion.

Here are some use cases for offline editing:

  1. Hardware engineer Jane goes on a 10-day field test. She takes a copy of the wiki with her for reference. During the test, she makes several observations that result in edits to pages in the wiki. Rather than waiting for Internet access, she revises those pages in the field. Upon returning to the office, she syncs her edits to the "master" wiki.
  2. Astronaut instructor John takes a mobile copy of the wiki with him to a classroom with no wi-fi access. During the class, he notices an error and makes the correction. When he returns to his office later in the day, he syncs his revision with the "master" wiki. A few days later, he has a similar situation while he attends a meeting offsite and doesn't have VPN access. So he makes his changes and then syncs them after getting back to his office.
  3. Astronaut Peggy is on the International Space Station. She has intermittent Internet access. In order to ensure she always has all the information she needs in the case of a contingency, she has a clone of the wiki onboard. As engineers and flight controllers make updates to the "master" wiki at NASA, these changes are periodically synced up to ISS as comm is available. As Peggy spends time on orbit and learns nuances of the hardware, she edits some wiki pages to share this knowledge with MCC. These edits must also be synced as intermittent comm is available.

These are some very NASA-centric scenarios, but I think they could easily be applied to engineers, doctors, humanitarians, etc.

TheDJ added a subscriber: TheDJ.Sep 27 2015, 10:13 AM

Slightly related to offline editing might be this ticket, for local autosaving of wikitext edits: https://gerrit.wikimedia.org/r/5130

Qgil added a comment.Oct 3 2015, 8:57 PM

Congratulations! This is one of the 52 proposals that made it through the first deadline of the Wikimedia-Developer-Summit-2016 selection process. Please pay attention to the next one: > By 6 Nov 2015, all Summit proposals must have active discussions and a Summit plan documented in the description. Proposals not reaching this critical mass can continue at their own path out of the Summit.

greg removed a subscriber: greg.Oct 4 2015, 2:07 AM
Qgil added a comment.Oct 28 2015, 10:13 AM

Considering that T113004: Make it easy to fork, branch, and merge pages (or more) is blocking this task and it is also being proposed as a Summit proposal, do you think it would be better to focus efforts on that task? I just fear spreading the attention to much, and not reaching to any conclusions at the end.

awight added a comment.EditedOct 29 2015, 7:06 AM

@Qgil
I would second that! T113004 makes sense as a first step, plus it enables many other new features beyond offline editing. Thanks for the thought and energy you're putting into organizing the summit work!

Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptNov 4 2015, 11:56 AM
Krinkle renamed this task from Enable off line editing (particularly for mobile) to support people with intermittent internet access to Enable offline editing (particularly for mobile) to support people with intermittent internet access .Nov 4 2015, 9:39 PM
Nemo_bis added a subscriber: Nemo_bis.EditedNov 7 2015, 8:25 AM

It sounds unlikely that T113004 really blocks this task. This task is rather self-contained, while T113004 is a total revolution (if not destruction) of the wiki system.

I also think that the proposed audience is wrong. The correct audience to address is users who don't have (practically) *any* connection, for instance Kiwix users.

cscott added a comment.Nov 7 2015, 1:56 PM

I agree with @Nemo_bis that T113004 is not really a blocker (although I don't agree with him that T113004 is a "destruction" of the wiki system, of course).

It would be more accurate to say that both T113004 and this task are blocked by a third task: better merge tools.

But for offline editing those tools don't even have to live in mediawiki-core; they could be part of the offline codebase.

a third task: better merge tools

Very true and that's IMHO in the top 5 of the most important things to fix in MediaWiki core. Better merge, diff and edit conflict resolution are tightly related I think, so I added most known issues to T72163.

Nemo_bis updated the task description. (Show Details)Apr 5 2016, 2:09 PM
Qgil removed a subscriber: Qgil.Apr 5 2016, 2:10 PM

@aripstra

Is this ticket still relevant? Thanks!

I am removing design research from this task since we don't have plans to work on it this quarter. Please let us know if and when this comes up - we are very interested in any work being done on mobile editing.

Nemo_bis renamed this task from Enable offline editing (particularly for mobile) to support people with intermittent internet access to Offline editing to support people with intermittent or no internet access (e.g. Kiwix, mobile).Jul 31 2016, 6:57 AM
Nemo_bis triaged this task as Normal priority.
Nemo_bis removed aripstra as the assignee of this task.
Nemo_bis updated the task description. (Show Details)
Jdlrobson moved this task from Needs triage to Triaged on the Mobile board.Jul 19 2017, 1:23 AM