Page MenuHomePhabricator

[Spike] Figure out Wiktionary API requirements/support
Closed, ResolvedPublic

Description

In order to complete our goal of having pop-up definitions of highlighted words from Wiktionary, we need to understand what kind of API functions we'll need to use, and what kind of extra functionality and/or workarounds we'll need to implement.

We'll need to answer at least the following questions:

  • Does Wiktionary have a "mobileview" api that returns the page broken down by sections?
  • Is the content of a Wiktionary page suitable for native rendering, or will we need to use a WebView?
  • Is the page content consistent enough that we don't need to do a whole lot of transforms on the content?
  • Does all of the above hold for non-EN wiktionaries?

Event Timeline

Dbrant created this task.Oct 2 2015, 4:53 PM
Dbrant raised the priority of this task from to Normal.
Dbrant updated the task description. (Show Details)
Dbrant moved this task to Next Sprint on the Wikipedia-Android-App-Backlog board.
Dbrant added a subscriber: Dbrant.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 2 2015, 4:53 PM

Does Wiktionary have a "mobileview" api that returns the page broken down by sections?

Yes, you can use nearly identical queries. However, note that the page casing may be surprising. For example, a "dog" page exists but a "Dog" page does not. i.e., the following query works:

https://fr.m.wiktionary.org/w/api.php?action=mobileview&format=json&formatversion=2&prop=text|sections|languagecount|thumb|image|id|revision|description|lastmodified|normalizedtitle|displaytitle|protection|editable&onlyrequestedsections=1&sections=3&sectionprop=toclevel|line|anchor&noheadings=true&page=dog&thumbsize=640&redirects=true

But the same query with page=Dog does not.

Is the content of a Wiktionary page suitable for native rendering, or will we need to use a WebView?
Is the page content consistent enough that we don't need to do a whole lot of transforms on the content?

I did some research on these questions. I'm a little bit excited because I think we might have an approach that could be practical and I think there's a big want for it in the community but no current solution.

With Parsoid, it's still going to be messy. We have to build a semantic definition selector for all of the noun / verb / adjective / etc templates for each language we want to support and take the first match to retrieve the semantic definition. So, for example, to pull the noun definition for banana in English or Spanish:

document.querySelectorAll('[data-mw*="en-noun"],[data-mw*="gl-noun"],...,...,...')

This selector would be expanded to includes verbs, adverbs, adjectives, etc, but could be targeted for the specific wiki. I think there will be a lot of finagling even after building out that massive list of localized templates but it really might work. If we obtain definitions in this fashion, I think we can subset the HTML used and render it with native views. We should engage @GWicke early to understand what Parsoid's plan is for localized templates and try to build something that can gracefully evolve to it.

If we don't do something akin to the above, I think we'll have to use a WebView and display the first five or so sections of content which hopefully includes the word definition. I think it will look very non-native even for a WebView to the point of being an anti-feature. I don't recommend this approach.

Does all of the above hold for non-EN wiktionaries?

From a spot check, yes. As a bonus, foreign word look ups give definitions when they're defined. That is, if you're reading a German article, you want to use the German dictionary. However, if you're reading an German article, and query melocotón (Spanish for peach) or chuối (Vietnamese for banana), you can still use the German dictionary and it will return the word if it has been defined for German.

Please see my note about localized templates though.

Dbrant closed this task as Resolved.Oct 26 2015, 3:17 PM