Page MenuHomePhabricator

When switching languages on mobile web, allow search for a language in any language
Closed, DeclinedPublic5 Story Points

Description

As a Spanish user viewing the English version of Wikipedia
I want to switch to the French version in my local language via typing francés which currently matches no results
so that I save time by not being required to remember that I need to search for it as "francais" or "french".

Related: T139799

More detail

Do this in beta first, ensuring that it's possible to later move this to stable.

Ideally, in addition to searching in your current language (T139799), I should be able to search in any langauge: "Francés" (spanish version), "Französisch" (german version), etc. This is currently possible using the Universal language selector and I believe they are simply using the action api: action=languagesearch to do so.

While a search request is pending, do not modify the result list. Only upon response modify the list.

Technical details

Change client side filtering to:

  1. API query example: https://en.wikipedia.org/w/api.php?action=languagesearch&format=json&search=aleman
  2. After that, match the language codes with the ones we have on the client to show the relevant results

AC

  • Searching for "aleman" in an english wiki with english language browser will show "German" in the list of languages.
  • While searching there's an unobtrusive loading indicator as showed in the mocks after 2 seconds of loading time.
  • If the api query fails, there is an error message shown as specified in the mocks `TBD``` @Nirzar.
  • If the api query takes too long (10s for example), it should timeout and there should be an error message shown as specified in the mocks.
  • If there are concurrent searches the results shown will be the ones of the last query issued. Previous old queries will be dismissed if they arrive later.
  • Search API querying (input changes) is debounced to avoid excessive network calls.
  • Unit and browser tests are adapted to the new behavior.
  • Existing event logging behaviour is unchanged
  • Event logging is updated to include languagesearch API request errors

Final mocks

Initial state

Successful search (results before 2 seconds) we don't want to show flickering spinner

If the api takes more than 2 seconds to return, then only we show the spinner


Notice the list is unaffected while the query is taking more than 2 seconds.

After 10s waiting (timeout), show error state:

Both:

  • I searched for a language that does not exist at all on wikipedia
  • I searched for a language that does exist on wikipedia but this article does not exist in that language

Suggested testing, RTL+LTR, landscape+portrait:

  • Android 2.3 Browser phone form factor
  • Android 4.x or higher Chrome phone form factor
  • Android 4.x or higher Chrome tablet form factor
  • iOS 9.3 iPhone Safari
  • iOS 9.3 iPad Safari
  • Opera Mini non-compressed (not extreme mode) Android 4.x or higher phone form factor
  • UC browser (not mini with compression) Android 4.x or higher phone form factor
  • Windows 7.5+ Phone IE
  • Desktop Firefox
  • Desktop Chrome
  • Desktop Safari
  • Internet Explorer 11
  • Edge

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Ellis added mocks: M167: combine saved + history into "my pages", M166: Wikimedia UI (Experimental Version), M168: Wikispeech player, M169: Wikispeech user settings, M170: Add support for colored background in terminal - screenshot, M171: Revision Slider Design Review 1, M172: too much whitespace, M173: Mobile app badges on the Portal page, M175: Hebrew banner, M174: Debug toolbar prefixed with timestamps, M74: User education inside Visual Editor, M75: MultiSelectWidget, Restricted Mockup, M77: Extension:SmiteSpam UI update, M78: Wikistats, M79: Android navigation drawer, M80: Collecting feedback in VE, M81: SOPA on lynx, M82: Color palette (Wikimedia Design Style Guide/WikimediaUI), M83: Feed styling spec, M84: Search in iOS, M85: Search experience, M86: Article specification, M87: Strange gate-and-submit pipeline manager behavior, M88: Screenshot of effort microsurvey, M89: blocked images in Iran, M90: Math dialog, M91: iOS specific icons for Wikipedia, M92: phab search, M93: Browser tests Jenkins jobs, M94: Shubham gujar, M95: First run for iOS app, M96: Math dialog, M98: Tabbar spec, M97: VE mobile second iteration , M99: 404 page layout, M100: Random card interaction, M101: WikimediaUI, M102: iOS Recent spec, M104: Picture of the day & Language selection, M105: Link preview for pre 6s, M106: echo for user:Base on 2015-10-24 in ukwiki, qqx interface language, M107: Single edit tab idea, M109: Support Wikipedia menu in Navigation drawer, M110: Overflow menu for cards, M108: article footer, M111: Screenshot of AutoWikiBowser rev 11727 while working on English Wikipedia, M112: Screenshot of AutoWikiBowser rev 11727 while working on English Wikipedia, M113: In the News - ios feed, M114: Nearby update, M115: Delete saved pages, M116: Change languages while searching, M118: Android bottom sheets, M119: Fan Monday, M117: Contents, M120: Notification across iOS, M121: PageImage displaying wrong image, M122: Drawers on iOS, M123: Reference drawer, M124: Share on iOS, M125: User pages, M126: Userpages on Mobile, M127: Native bar spec - android, M128: Ascii PAWS logo, M129: A try to add user AS in a Herald rule by Base, M130: iOS feed loading, M131: Related articles - responsive, M132: (Outdated) Notifications catalog, M133: Wtf banner, M134: MOUAD MANSOURI, M135: VisualEditor anon welcome popup , M136: AutoWikiBrowser rev 11772 skip screen, M137: Black Echo text in a read notification, M138: Template selection, M139: Release Engineering, M140: Android about page, M141: Refresh saved pages, M144: android article actions, M143: Phlogiston, M142: https://en.wikipedia.org/wiki/Ezhil_%28programming_language%29, M145: Managing a collection, M147: A badge of honorable shame, M148: I7ef10af3c2e379e51e9dde588fc578c9a3f37ef1, M149: language overlay, M150: Add page to collection, M151: Collections onboarding, M152: Smoke + parallel strategy for running end-to-end tests, M153: Save prompt, M154: My dashboard, M155: Feed wireframe, M156: missing border, M157: TOC, M158: Bad scores for ptwiki, M159: newsletter extension - list, M160: newsletter extension - details, M161: Text resize , M162: 283120,7 comparison, M163: Language settings, M164: Mathoid rendering on Mac, M165: lead image selection.Jun 13 2016, 3:54 AM

@Aklapper I apologize if you are not the Phabricator police, but the individual who just added a huge number of assets to this task might be acting in good faith and need help or might be acting with bad intention.

JKatzWMF removed mocks: M165: lead image selection, M164: Mathoid rendering on Mac, M163: Language settings, M162: 283120,7 comparison, M161: Text resize , M160: newsletter extension - details, M159: newsletter extension - list, M158: Bad scores for ptwiki, M157: TOC, M156: missing border, M155: Feed wireframe, M154: My dashboard, M153: Save prompt, M152: Smoke + parallel strategy for running end-to-end tests, M151: Collections onboarding, M150: Add page to collection, M149: language overlay, M148: I7ef10af3c2e379e51e9dde588fc578c9a3f37ef1, M147: A badge of honorable shame, M145: Managing a collection, M142: https://en.wikipedia.org/wiki/Ezhil_%28programming_language%29, M143: Phlogiston, M144: android article actions, M141: Refresh saved pages, M140: Android about page, M139: Release Engineering, M138: Template selection, M137: Black Echo text in a read notification, M136: AutoWikiBrowser rev 11772 skip screen, M135: VisualEditor anon welcome popup , M134: MOUAD MANSOURI, M133: Wtf banner, M132: (Outdated) Notifications catalog, M131: Related articles - responsive, M130: iOS feed loading, M129: A try to add user AS in a Herald rule by Base, M128: Ascii PAWS logo, M127: Native bar spec - android, M126: Userpages on Mobile, M125: User pages, M124: Share on iOS, M123: Reference drawer, M122: Drawers on iOS, M121: PageImage displaying wrong image, M120: Notification across iOS, M117: Contents, M119: Fan Monday, M118: Android bottom sheets, M116: Change languages while searching, M115: Delete saved pages, M114: Nearby update, M113: In the News - ios feed, M112: Screenshot of AutoWikiBowser rev 11727 while working on English Wikipedia, M111: Screenshot of AutoWikiBowser rev 11727 while working on English Wikipedia, M108: article footer, M110: Overflow menu for cards, M109: Support Wikipedia menu in Navigation drawer, M107: Single edit tab idea, M106: echo for user:Base on 2015-10-24 in ukwiki, qqx interface language, M105: Link preview for pre 6s, M104: Picture of the day & Language selection, M102: iOS Recent spec, M101: WikimediaUI, M100: Random card interaction, M99: 404 page layout, M97: VE mobile second iteration , M98: Tabbar spec, M96: Math dialog, M95: First run for iOS app, M94: Shubham gujar, M93: Browser tests Jenkins jobs, M92: phab search, M91: iOS specific icons for Wikipedia, M90: Math dialog, M89: blocked images in Iran, M88: Screenshot of effort microsurvey, M87: Strange gate-and-submit pipeline manager behavior, M86: Article specification, M85: Search experience, M84: Search in iOS, M83: Feed styling spec, M82: Color palette (Wikimedia Design Style Guide/WikimediaUI), M81: SOPA on lynx, M80: Collecting feedback in VE, M79: Android navigation drawer, M78: Wikistats, M77: Extension:SmiteSpam UI update, Restricted Mockup, M75: MultiSelectWidget, M74: User education inside Visual Editor, M174: Debug toolbar prefixed with timestamps, M175: Hebrew banner, M173: Mobile app badges on the Portal page, M172: too much whitespace, M171: Revision Slider Design Review 1, M170: Add support for colored background in terminal - screenshot, M169: Wikispeech user settings, M168: Wikispeech player, M166: Wikimedia UI (Experimental Version), M167: combine saved + history into "my pages".Jun 13 2016, 4:24 AM
Jhernandez added a subscriber: Jhernandez.

Currently MobileFrontend filters locally on the client, without api queries.

This will change so, so we need Design input about the loading and failed states

jhobs triaged this task as High priority.Jun 15 2016, 3:33 PM
jhobs moved this task from To Triage to Needs Analysis on the Readers-Web-Backlog board.
jhobs added a project: MobileFrontend.

@Nirzar, to clarify, the point is to be able to use the keyboard to enter any conceivable language, only keeping the records for languages that actually matched. Is that right?

In other words, the default, unfiltered language list in the modal will remain as is, but upon search will be filtered. It's just that the search could match multiple spellings of the language name.

@dr0ptp4kt that is correct. for example, i search "Spanish" but the result shows "Español"

Okay, @Nirzar. So the cell from the overlay will look the same, it's just that different search strings will pull it up, whereas you had to be more specific to match the spelling in the cell.

@dr0ptp4kt yes. its not only specific but you had to have the keyboard too. for example you cant type "hindi" to search for hindi. you had to write it as "हिंदी"

as a user, i might be learning to read a language but not write it.

dr0ptp4kt updated the task description. (Show Details)Jun 30 2016, 4:00 PM
MBinder_WMF changed the task status from Open to Stalled.Jun 30 2016, 4:30 PM
MBinder_WMF added a subscriber: MBinder_WMF.

@dr0ptp4kt and @JKatzWMF to get together to flesh this out more, address technical concerns, possibly create subtask spike.

Investigating, if we only want to return the language of the users current language, it seems we can simply modify the existing request so that it passes llinlanguagecode
https://en.wikipedia.org/wiki/Special:ApiSandbox#action=query&format=json&prop=langlinks&meta=siteinfo&titles=Barack+Obama&llprop=autonym%7Clangname&llinlanguagecode=en&lllimit=500

Nirzar added a comment.Jul 1 2016, 6:31 AM

@Jdlrobson as we talked in the meeting i think if clientside search is possible to fix we should go for it. we can also check the client side logic on iOS which works

from design, the requirement seems pretty clear, if it needs more explanation or any open questions let me know.

Jhernandez changed the task status from Stalled to Open.Jul 8 2016, 10:41 AM
Jhernandez added a comment.EditedJul 8 2016, 10:50 AM

To be clear, this task makes searching languages change from instant and synchronous (Search->Show results) to be asynchronous and needing the network (Loading->Get results->Show results or error).

So @Nirzar as I mentioned in T137680#2382749 we need mocks for how the loading and error states should look on that modal. I'm guessing we don't want the same spinner blanks results list and flashes stuff that we do when searching pages from the top bar.

Without those, the task won't be able to be properly estimated or be ready for working on it, so we'll have to postpone it after 77. In theory we estimate next Monday (9amPST) and on sprint kickoff, and can do emailstimation if needed so during next week, so it should be fine if the mocks are posted soon this next week.

Given this may be a thick task, the sooner we get to estimating it, the better.

cc/ @dr0ptp4kt

Jhernandez updated the task description. (Show Details)Jul 8 2016, 11:01 AM

@Nirzar I've updated the description with fleshed out AC, have a look, it should help you see the 2 or 3 states that need mocks. Thanks!

Jdlrobson updated the task description. (Show Details)Jul 8 2016, 9:42 PM

There are 4 states needed for this ticket.

1. I'm searching for a language in any other language and script AND I have fast internet

Step 1

Step 2

in this case, the api will return the results quickly, and we don't want to show flickering spinner. so if the api takes more than 2 seconds to return then only we show the spinner.

that is
2. I'm searching for a language in any other language and script AND I have slow internet/or server is slow
Step 1

Step 2


Notice the list is unaffected while the query is taking more than 2 seconds.

3. I searched for a language that does not exist at all on wikipedia

4. I searched for a language that does exist on wikipedia but this article does not exist in that language

NOTE: currently we do not handle case 3 and case 4. so we can keep it as good to have. handling those cases is definitely the way to go since communication is very important here.

As @phuedx points out we need to think about search on slower connections T137068
Nirzar to design prototype of behaviour and update/link from description.
Initial estimates by entire team ranged from 3 to 8.

phuedx removed Nirzar as the assignee of this task.Jul 11 2016, 5:00 PM
phuedx updated the task description. (Show Details)
Nirzar claimed this task.Jul 18 2016, 6:59 PM

@Nirzar Could we remove 4 in favor of 3 (to avoid the different error message, in 4 is problematic because the search may return several languages)

@Nirzar The only thing I see missing is the error state (like 3) but for when there is a timeout (of 2 secs as you said).

@Nirzar Sorry, I misunderstood.

So after 2 secs, we'll show a spinner to show it's taking time.

But what happens after a longer timeout? Like 10 seconds? I believe we should show some kind of error state like 3 saying something like "Sorry. The search is taking too long. Retry" or something along the lines

Jhernandez updated the task description. (Show Details)Jul 20 2016, 4:33 PM

(Likewise, I would classify timeout messages along the same lines as a legitimate network error, for example. It would be nice if we had one error message for timeouts/network error.)

Nirzar added a comment.EditedJul 20 2016, 9:36 PM

@Jhernandez

Could we remove 4 in favor of 3

yeah, i thought so. we can make it a generic message that works if language doesn't exists or exists AND article is not available in that language.

this ->

what happens after a longer timeout?

sure, we can do another error state in case of failure

Thanks @Nirzar. I'm going to add them together in the description

Jhernandez updated the task description. (Show Details)Jul 21 2016, 11:15 AM
MBinder_WMF set the point value for this task to 5.Jul 25 2016, 4:19 PM
dr0ptp4kt updated the task description. (Show Details)Jul 30 2016, 2:59 AM

"takes more than 2 seconds" - what does this refer to ? 2g connection? wifi?

@Nirzar ^ would you please answer that question?

As I understand it, though, that means that if the server hasn't responded within 2 seconds since the user's query was sent, initiate the spinner.

Nirzar added a comment.Aug 3 2016, 5:36 PM

@Jdlrobson yes. it's 2 client seconds. irrespective of internet speed

The task is on hold until I and @Jdlrobson have a conversation as was suggested by Jon.

@bmansurov and I talked about this and we identified 5 patches:

  1. Error message when no languages found in search
  2. LanguageSearchGateway and unit tests
  3. Async behaviour in LanguageOverlay (use LanguageSearchGateway to do search)
  4. Behaviour to show "Failed to look up languages"
  5. Add EventLogging for errors

@dr0ptp4kt it's not clear how https://meta.wikimedia.org/wiki/Schema:MobileWebLanguageSwitcher should incorporate errors in search. I guess this will involve adding a new "error" to the enum "event" property and a new optional errorMsg property? Can you clarify. Patch #5 is blocked on this.

It's also not clear if this should be beta only to start with. Given the risk of changing behaviour so soon to deployment (we are deploying the language switcher this sprint), beta seems to be the correct place for this but this should be added to description. Patch #3 is blocked on this. Thanks.

Nirzar added a comment.Aug 3 2016, 6:48 PM

@Jdlrobson will it possible to test this with let's say more than 100 languages? i want to see how the search behaves on slower networks. and see all the cases in action. on our test environment we have only 5-10 languages.

bmansurov claimed this task.Aug 3 2016, 6:50 PM
bmansurov moved this task from To Do to Doing on the Reading-Web-Sprint-78-Terminal-Velocity board.

I'm going to start with "1. Error message when no languages found in search".

@Nirzar I'm not sure I follow your intentions.

The number of languages in the list will not change.
https://en.m.wikipedia.org/wiki/Barack_Obama#/languages
The only thing that introduces a delay here is how fast the search API is. You can test that in production right now: https://en.wikipedia.org/w/api.php?action=languagesearch&format=json&search=aleman - that takes 626ms even on gprs.

There are a few things I don't think we have thought through well:

  1. If I search for in e.g. indonesian then English is now a valid search result as in Uzbek English is "ingliz". However the sorting is still done based on the names in the current language so English will appear above Indonesian despite Ingliz being after Indoneziya in Uzbek.
  2. The async behaviour will cause a re-render of results. It's hard to tell how jarring that might be. For instance if I search for "in", as in the above example, indonesian is the 3rd result in the synchronous search. If we hit the API after 600ms English and who knows how many other languages will magically appear above.

I'm actually quite concerned this will make search less useful for some as now they have to type more to filter, so I would strongly recommend doing this in beta first (or even better prototyping) before pushing this out in front of people.

...
@dr0ptp4kt it's not clear how https://meta.wikimedia.org/wiki/Schema:MobileWebLanguageSwitcher should incorporate errors in search. I guess this will involve adding a new "error" to the enum "event" property and a new optional errorMsg property? Can you clarify. Patch #5 is blocked on this.

I apologize, that acceptance criterion escaped my attention. I believe what @phuedx was going for here (see https://phabricator.wikimedia.org/transactions/detail/PHID-XACT-TASK-mixbliobc4xx5up/) is that there's a non-zero chance the languagesearch endpoint will return application layer failures or may timeout or that sort of thing, and it's important to have awareness for this. Accordingly, yes, please do that very thing you suggest with "error" as a potential action and "errorState" to contain the error source; as we realized with Popups, we shouldn't log an error if our JavaScript intentionally cancelled an API call (e.g., user was typing search fast), but rather when "real" errors occur.

It's also not clear if this should be beta only to start with. Given the risk of changing behaviour so soon to deployment (we are deploying the language switcher this sprint), beta seems to be the correct place for this but this should be added to description. Patch #3 is blocked on this. Thanks.

I agree with your risk assessment. Please do this in beta first; the code should be structured as much as possible to make it easy to move it from beta to stable, naturally. Would you please also open a beta-to-stable follow up task with the appropriate level of detail? I know sometimes transition from beta to stable is more than meets the eye even if we're trying to make it simple up front to do it.

dr0ptp4kt updated the task description. (Show Details)Aug 3 2016, 7:19 PM

I updated the task Description to refer to beta.

dr0ptp4kt removed bmansurov as the assignee of this task.Aug 3 2016, 8:04 PM

After further discussion, we're putting this task on hold and removing it from the sprint.

The existing language search, which is already supports searching in the current wiki language plus the destination's canonical spelling of the language, coupled with the fact that if the user's keyboard language has a translation it will be floated to preferred languages, provides ample coverage for the common cases. The relative risk of poor connections mobile also poses a concern for getting the search to be fast enough (n.b., this isn't as big of a concern on desktop generally, where connections tend to be faster).

@dr0ptp4kt should we abandon this ticket?

dr0ptp4kt closed this task as Declined.Aug 25 2016, 8:49 PM

Yeah, for now. Marking as declined. We can bring this back in the future if we see a strong reason to do so.

Restricted Application removed a subscriber: Liuxinyu970226. · View Herald TranscriptAug 25 2016, 8:49 PM