Page MenuHomePhabricator

[Task] Functionality: Play selected text (Wikispeech)
Closed, ResolvedPublic4 Estimated Story Points

Description

Add possibility to play only selected text.
Existing: No
To do: Make sure player can start and stop at selected text.

Identified as a component during the pilot study.

While this might not be dependent on T140105 it is preferable to have that done first to determine what (if anything) to re-use.

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Worked on in Wikispeech (Sprint 2016-11-30):

  • Investigated which parts of the code base for this are also needed for T148622 and T148623

To do in Wikispeech (Sprint 2016-12-14):

  • The relevant code will be reused for T148622 and T148623 but this task is put on hold since it is not prioritised for T151786.
Sebastian_Berlin-WMSE changed the point value for this task from 3 to 12.Feb 7 2017, 2:45 PM

Changed story points to new system

I have tried stopping playback using the timeupdate event. This kinda works, but isn't very accurate; the next token (several if their short enough) is often played.

By experience from T148623, setTimeout should work better than the timeupdate event.

Worked on in Wikispeech (Sprint 2017-04-25):

  • Playback of a single part of an utterance works, assuming it has been prepared.

To do in Wikispeech (Sprint 2017-05-10):

  • Playback across multiple utterances.
  • Selecting outside/between words.
  • Playback of unprepared utterances.
  • Issues with highlighting words and selection.
  • Ensure that there is a (separate) task for playback of text outside of mw-body-content.
Sebastian_Berlin-WMSE raised the priority of this task from Lowest to Low.Sep 20 2017, 1:55 PM
Sebastian_Berlin-WMSE raised the priority of this task from Low to Medium.Sep 22 2017, 10:22 AM

Since the highlighting interferes with the selection (by adding elements), I decided to first implement this with highlighting disabled. After a few snags, including a bug where tokens didn't have the correct offsets if they appeared in other text nodes than the start of the utterance and the fact that some element (e.g. the edit section span) caused multi range selections because they're not selectable, this seems to work.

Nice! It would be great to show that the words selected are the ones that will be played. Having an icon appearing next to the marked text would make it easier to understand the logic. Also, please investigate if it is possible to have the words marked highlighted in a new color.

I did a mockup of how different ways highlighting selection could be done: https://jsfiddle.net/Beppe/fjfs7kvq/. My initial opinion is that either #3 or #4 best shows what's going on.

I looked at Readspeaker and Texthelp, and they both use #3.

The current implementation for when to start and stop playing (using events and timeouts) isn't very exact; Often a split second of the audio is played before the start time is set and sometimes the audio stops too early. To fix this we'll need some kind of library, e.g. Howler.js

After a quick test, Howler.js also suffers from sometimes stopping too early.

I've noticed that the timing issues aren't always that noticeable, so I think that can wait until it's an obvious issue. There may also be a way to get better stop times by using the timers already in place for token highlighting. These trigger each token, so setting a timer for the last token should give an OK timing for stopping.

Change 385373 had a related patch set uploaded (by Sebastian Berlin (WMSE); owner: Sebastian Berlin (WMSE)):
[mediawiki/extensions/Wikispeech@master] Recite selected text

https://gerrit.wikimedia.org/r/385373

Implemented, but doesn't work with highlighting (it's disabled when text is selected). Floating play button when text is selected will come in a separate patch.

Forgot to test this with Arabic, which doesn't work. This seems to be because the response from the TTS server contains the input string with diacritics added and not the original. The latter is needed to match the tokens to the text.

I've now implemented the selection player locally. It works for most cases I've tried, but there are some compromises:

  1. The selection only gets played if start and end of selection is in an utterance. That means, if the selection starts/ends in an element that isn't recited (e.g. reference link), the player will recite the whole article.
  2. There is a slight, but noticeable, delay when a selection container (node in which start or end of selection is) is an element. At this point, some calculation is needed to get the actual text node that is selected. Typically, this happens when triple clicking to select a whole paragraph. On my machine, the delay is about 150 ms.
  3. Selection is not valid when when start/end is between two paragraphs, if the first paragraph ends with an unrecited element.

1 would need some discussion about how this should be handled (should (all) text that is normally not recited be recited when explicitly selected? If not, is it OK to start reciting mid selection?). 2 and 3 are more technical, but would still need some work to fix. While all three are obvious shortcomings, I feel that they're not show stoppers at this point, especially if we add them to a list of known bugs.

Floating play button when text is selected will come in a separate patch.

This ended up in the same patch instead, since there were some changes to the original patch too.

Please create tasks for things you actively chose not to implement as part of this patch. Stick them either as known-bugs or enhancements on Wikispeech-WMSE,

Sebastian_Berlin-WMSE changed the point value for this task from 12 to 4.

Change 385373 merged by jenkins-bot:
[mediawiki/extensions/Wikispeech@master] Recite selected text

https://gerrit.wikimedia.org/r/385373

Will soon be up and running on our test wiki.