Page MenuHomePhabricator

Improve utterance storage
Closed, ResolvedPublic

Description

As was pointed out by @Bawolff in this discussion, the current implementation (postentially) adds a lot of HTML. At the moment, all cleaned text is added again, under the utterances-element. While this may be necessary when a client doesn't use JS, it would be good to avoid when possible. It would also make further development easier, as working with HTML elements as "data containers" can be quite cumbersome.

Decision

Ajax request to action API

Reasoning

  • Making API requests to cleaning and segmenting allows easy re-use
  • Storing the data as a JS-object makes handling easier
  • Ajax request(s) removes the need to store this in the html
  • If a user does not use Wikispeech the process will not have to be run.

Known Trade-offs

  • Likely harder to implement solution for No-JS
  • No pre-loading of first utterance (at least not as default)

Follow-up tasks

  • Implement new utterance storage
    • Build API module
    • Request to API
    • Update existing code to use new utterance storage
  • Implement No-JS workaround

Options

  1. AJAX:
    • Pros:
      • Seems to be designed for things like this.
    • Cons:
      • ...
  2. Streamlining HTML: If no other solution is found, it should still be possible to decrease the amount of HTML that needs to be added, e.g. by storing string positions, rather than the strings themselves.
    • Pros:
      • Most of current implementation of preprocessing can be kept as it is.
    • Cons:
      • Still adds HTML.

Event Timeline

Braindump:
Still inject into html but only after the page has been loaded and something triggers the need. We should be able to use a hook to trigger the same process as today. Non-js triggers could be:

  • url parameter
  • button/link embedded during page creation (very small html&limit change)
  • user setting (cannot be the only way since we loose all anonymous users)

Downside is that this would likely require athe least a partial reload of the page.

I've experimented with an AJAX solution (I think, I'm still not entirely sure of the definition) that seems to work.

I've added a prop to action=query which gets the HTML (using a FauxRequest for action=parse) and then runs the clean-segment-chain (HTML no longer needed). The resulting utterances is then returned as JSON.

The JS now starts by getting the utterances through the API and stores them as a JS object. This is then used instead of the utterances-element. I've so far hacked together a solution where play/stop works, except for some strangeness with requesting new utterances (two are requested at a time).

All in all, I think this looks promising. It will take some reworking of the JS code to get everything in place, but when that's done, I think it will make things easier in the future. Also, as a side effect, having cleaning and segmenting as part of the API will make it very easy for others to use.

True. The actionApi was not part of our original plan but it would make a lot of sense to expose e.g.utterances to anyone who wants to use it for other purposes.

There is some relatively simple example code in OpenBadges for new API components.

Note from offline discussion:
For No-JS usage: url parameter which triggers API call (or the underlying method) via PHP and injects html into the page.
Likely implementation would be play buttons per paragraph or similar.
This should likely be broken out as a follow-up task.

Lokal_Profil updated the task description. (Show Details)
Lokal_Profil set the point value for this task to 1.
Lokal_Profil moved this task from Backlog to In progress on the Wikispeech (Sprint 2017-01-25) board.

Left in order to resolve:

  • Create follow-up tasks with story point estimates.

I've experimented with an AJAX solution (I think, I'm still not entirely sure of the definition) that seems to work.

I've added a prop to action=query which gets the HTML (using a FauxRequest for action=parse) and then runs the clean-segment-chain (HTML no longer needed). The resulting utterances is then returned as JSON.

The JS now starts by getting the utterances through the API and stores them as a JS object. This is then used instead of the utterances-element. I've so far hacked together a solution where play/stop works, except for some strangeness with requesting new utterances (two are requested at a time).

All in all, I think this looks promising. It will take some reworking of the JS code to get everything in place, but when that's done, I think it will make things easier in the future. Also, as a side effect, having cleaning and segmenting as part of the API will make it very easy for others to use.

If you're storing it in a page_prop (Or anything else that comes from the db), keep in mind the version in the database will usually be the canonical version, but the user may not be viewing the canonical version of the page (e.g. Due to language settings or what not). There's also the possibility of race conditions when using page_props.

Sebastian_Berlin-WMSE changed the point value for this task from 1 to 4.Feb 7 2017, 1:07 PM

This will be done after T151786: Publically accessible demo (player) [Stage 1+2] is finished.

Changed story points to 1 p = 1h.

Sebastian_Berlin-WMSE changed the task status from Open to Stalled.Feb 7 2017, 1:19 PM
Lokal_Profil changed the point value for this task from 4 to 14.Mar 8 2017, 10:23 AM
Lokal_Profil edited projects, added Wikispeech (Sprint 2017-03-08); removed Wikispeech.

Re-estimated story points to capture how deeply integrated this and the need to maintain a html fallback for noJS users (common in the target audience with particular need for Wikispeech).

Lokal_Profil changed the task status from Stalled to Open.Mar 8 2017, 10:23 AM

This task should result in a plan for how to implement this, including identified follow up tasks.

In Wikispeech (Sprint 2017-04-05):

  • Compile thoughts mentioned in this task (and tasks mentioned here).
Lokal_Profil removed the point value for this task.May 9 2017, 2:55 PM

Removed points to reflect that these are now held by the subtasks.

On completion of the subtasks ensure that there is nothing in this task that is left to resolve.

As the subtasks have been resolved