Page MenuHomePhabricator

SwitchControl users cannot open links in articles in iOS app
Closed, InvalidPublic

Description

It seems to be impossible to open any link inside an article using Switch Control in the iOS app. Executing the "Tap" action on any link does not open it but instead it just resets the Switch Control position to the "<Explore" back button in the navigation bar.

Switch Control is an iOS assistive technology for severely paralyzed people or people suffering similar disabilities who cannot control the device in traditional ways and must rely on control by a single switch or a small number of switches (e.g. pressing a switch by palm of hand, activating a switch by winking or tilting their head to a side).

Therefore, users relying on Switch Control won't be able to browse Wikipedia by following links in articles.

I made a test using a simple demo app with just the navigation bar, tab bar and a UIWebView with a Wikipedia article open in it and Switch Control did open links normally, so there must be something in the iOS app that prevents it from doing so.

Related Objects

Event Timeline

hhanke created this task.Jan 31 2016, 11:09 PM
hhanke raised the priority of this task from to High.
hhanke updated the task description. (Show Details)
hhanke added subscribers: BGerstle-WMF, hhanke, JMinor and 7 others.
Restricted Application added a subscriber: Danny_B. · View Herald TranscriptJan 31 2016, 11:09 PM
hhanke claimed this task.Feb 2 2016, 9:36 AM
Fjalapeno lowered the priority of this task from High to Low.Feb 5 2016, 5:50 PM

@Fjalapeno Since I don't know how and to which purpose exactly do you use priorities in this project, I might very well be mistaken. But it still seems to me that since this bug renders a core functionality unusable by a certain class of users, it seems strange to me that it should be classified as "Low".

I didn't set "High" myself, it was inherited from the supertask, but perhaps "Normal" would be more fitting.

This said, I am still working on understanding what is going on in this bug and how could we fix it. This is not an easy issue, particularly because this might be another instance where iOS itself is buggy. I'll let you know as soon as I have some news about what we could do about it.

The problem of hyperlink activation with Switch Control seems to be in the handling of events in the javascript listeners.js, which disables the standard onclick event in favor of custom handling of users touches in handleTouchEnded(). I have verified that taping a hyperlink inside the web view with Switch Control does reliably fire an onclick event on the link, but it doesn't fire the touchend event. Therefore, the onclick event is dismissed and nothing happens.

It seems to me we will have to respect the onclick event in some way and I would welcome your input here as to what you think would be the best way to do it. I am not sure what was the reason to use touchend instead of onclick and it also looks to me (just a quick impression now) that this might actually influence more things with Switch Control than just following hyperlinks. Perhaps because of force touch?

If onclick handling cannot be done for some reason or it is otherwise too difficult, there is also the option of behaving differently when Switch Control is running. We can check it in the Swift/Objective-C code and pass this information to the javascript. An idea for a hack would be that when we know that Switch Control is running, we could simulate a touchend event each time we detect an onclick event. But this should be last resort I think.

I would be very happy for your ideas :-)

(This also led me to discover a partial but extremely impractical workaround for Switch Control users. When instead of taping the item directly, one uses the Switch Control menu to navigate into Others -> Row 1 -> Gestures -> Row 1 -> Tap, then if one is lucky, it is possible to fire the touchend event and activate the link. But this is nearly impossible to discover for a user and also it is a very tiresome process for just activating a link. Also, I only say that it only works when one is lucky, because this does not really work for links that are split over two lines, as is often the case. When the link is split, the geometrical middle of the SwitchControl does not lie over the link but over further text and therefore the user does not really tap the link, but taps into further text. This may activate the desired link or another link based on which one utilities.findClosest() will find.)

@hhanke sounds... fun ;-). i'll have to do more research on switch control so i can better understand the normal behavior, and then figure out a way for it to coexist with our touch handling logic.

@BGerstle-WMF If you want to try it, I recommend to set Settings->General->Accessibility->Accessibility Shortcut to Switch Control (and only that). This will make it easy for you to turn it on and off using three quick presses of the home button.

I also recommend to set Settings->General->Accessibility->Switch Control->Speech to On. This will help you to understand what is going on, especially when the Switch Control cursor goes over multi-line links or into non visually visible places (which it shouldn't, but...).

JMinor raised the priority of this task from Low to High.Feb 9 2016, 12:01 AM

This was accidentally set to "Low" priority while being moved on our board. Re-upping it to High, as this is a fundamental aspect of the app which should be fully accessible.

@hhanke sorry misprioritized by mistake… Phabricator has a mind of its own sometimes.

Volker_E moved this task from Unsorted to iOS on the Accessibility board.Feb 11 2016, 10:51 PM
Danny_B removed a subscriber: Danny_B.Feb 11 2016, 10:52 PM

@BGerstle-WMF Investigating this issue more and giving it some more thought, I think we should observe the status of Switch Control in WMFArticleViewController or in a similar place. When Switch Control is running, we could set a preferOnclickEvents variable in javascript and then we can react to it when handling events. If preferOnclickEvents is true, we would maintain the preventDefault() behavior in document.onclick() but simulate a touchended event by calling touchEndedWithoutDragging() directly. I tested this and it seems to solve the original problem.

The issue here seems to be how to prevent an real touchended event from executing a duplicit action (or even possibly a different action) if it also comes. This would happen when we get both touchended and onclick events from Switch Control. This can happen, e.g. when the user chooses the other way to tap the link. Also, we cannot rely on the fact that the problematic Switch Control action we are trying to handle will never actually dispatch a touchended event as this is not documented by Apple.

One way to deal with this would be to do nothing in touchEndedWithoutDragging if preferOnclickEvents is true and the call to it is not simulated from the onclick handler. It seem to me that all the actions handled here are not specific to touch and the click behavior will suffice. But even if this assumption would turn out not to be entirely correct, it is hard to imagine that it could make behavior with Switch Control any worse than it is right now :-)

Would it be OK if I send a PR for this?

(Whether it is correct or a bug that Switch Control dispatches an onclick event instead of simulating a touchended event remains a mystery to me. It is clear that Switch Control actions are not true taps, e.g. they do not have a beginning and an end, but on the other hand developers perceive iOS as a touch only interface so there seems to be a conflict. Plus this seems to be different from how VoiceOver behaves. Probably we should file a ticket on Apple Bug Report too, just in case… )

@hhanke we are finalizing our release this week and I would love to get this fixed. Any chance you could submit that PR soon.

@BGerstle-WMF any issues with @hhanke proposed solution?

@JMinor Due to other work, I won't be able to look at it before the weekend, but maybe I could send something on Sunday if that would still be useful to you. This said, I would also be concerned about the possibility that such a last-minute modification at a key place might break something (although in theory it should only affect Switch Control). Do you still plan to do some significant testing after Sunday?

As an alternative, I would consider communicating to the users that accessibility is something that is currently being worked on and will really be there in a subsequent release. This would also cover the other issues we still have unresolved.

Please let me know what do you want to do. I will try to help of course :-)

Thanks @hhanke

I think Sunday would introduce too much risk, given that we're currently running regressions, etc.

I've retagged this for our next release, and I am definitely planning to make a public commitment that we will work through these issues on subsequent updates.

@JMinor OK, understood. Thank you again for thinking about these people!

JMinor removed hhanke as the assignee of this task.May 9 2016, 10:50 PM
JMinor lowered the priority of this task from High to Normal.
JMinor moved this task from Backlog to Needs Acceptance Criteria on the iOS-app-v5.1.0-Saturn board.
JMinor lowered the priority of this task from Normal to Low.May 24 2017, 6:54 PM
LGoto closed this task as Invalid.Mar 26 2019, 9:16 PM
dusek removed a subscriber: dusek.Mar 26 2019, 9:23 PM