@Aklapper I apologize if you are not the Phabricator police, but the individual who just added a huge number of assets to this task might be acting in good faith and need help or might be acting with bad intention.

Currently MobileFrontend filters locally on the client, without api queries.

This will change so, so we need Design input about the loading and failed states

@Nirzar, to clarify, the point is to be able to use the keyboard to enter any conceivable language, only keeping the records for languages that actually matched. Is that right?

In other words, the default, unfiltered language list in the modal will remain as is, but upon search will be filtered. It's just that the search could match multiple spellings of the language name.

@dr0ptp4kt that is correct. for example, i search "Spanish" but the result shows "Español"

Okay, @Nirzar. So the cell from the overlay will look the same, it's just that different search strings will pull it up, whereas you had to be more specific to match the spelling in the cell.

@dr0ptp4kt yes. its not only specific but you had to have the keyboard too. for example you cant type "hindi" to search for hindi. you had to write it as "हिंदी"

as a user, i might be learning to read a language but not write it.

Investigating, if we only want to return the language of the users current language, it seems we can simply modify the existing request so that it passes llinlanguagecode
https://en.wikipedia.org/wiki/Special:ApiSandbox#action=query&format=json&prop=langlinks&meta=siteinfo&titles=Barack+Obama&llprop=autonym%7Clangname&llinlanguagecode=en&lllimit=500

@Jdlrobson as we talked in the meeting i think if clientside search is possible to fix we should go for it. we can also check the client side logic on iOS which works

from design, the requirement seems pretty clear, if it needs more explanation or any open questions let me know.

To be clear, this task makes searching languages change from instant and synchronous (Search->Show results) to be asynchronous and needing the network (Loading->Get results->Show results or error).

So @Nirzar as I mentioned in T137680#2382749 we need mocks for how the loading and error states should look on that modal. I'm guessing we don't want the same spinner blanks results list and flashes stuff that we do when searching pages from the top bar.

Without those, the task won't be able to be properly estimated or be ready for working on it, so we'll have to postpone it after 77. In theory we estimate next Monday (9amPST) and on sprint kickoff, and can do emailstimation if needed so during next week, so it should be fine if the mocks are posted soon this next week.

Given this may be a thick task, the sooner we get to estimating it, the better.

cc/ @dr0ptp4kt

@Nirzar I've updated the description with fleshed out AC, have a look, it should help you see the 2 or 3 states that need mocks. Thanks!

There are 4 states needed for this ticket.

1. I'm searching for a language in any other language and script AND I have fast internet

Step 1

Step 2

in this case, the api will return the results quickly, and we don't want to show flickering spinner. so if the api takes more than 2 seconds to return then only we show the spinner.

that is
2. I'm searching for a language in any other language and script AND I have slow internet/or server is slow
Step 1

Step 2

3. I searched for a language that does not exist at all on wikipedia

4. I searched for a language that does exist on wikipedia but this article does not exist in that language

As @phuedx points out we need to think about search on slower connections T137068
Nirzar to design prototype of behaviour and update/link from description.
Initial estimates by entire team ranged from 3 to 8.

@Nirzar Could we remove 4 in favor of 3 (to avoid the different error message, in 4 is problematic because the search may return several languages)

@Nirzar The only thing I see missing is the error state (like 3) but for when there is a timeout (of 2 secs as you said).

@Nirzar Sorry, I misunderstood.

So after 2 secs, we'll show a spinner to show it's taking time.

But what happens after a longer timeout? Like 10 seconds? I believe we should show some kind of error state like 3 saying something like "Sorry. The search is taking too long. Retry" or something along the lines

(Likewise, I would classify timeout messages along the same lines as a legitimate network error, for example. It would be nice if we had one error message for timeouts/network error.)

@Jhernandez

Could we remove 4 in favor of 3

yeah, i thought so. we can make it a generic message that works if language doesn't exists or exists AND article is not available in that language.

this ->

what happens after a longer timeout?

sure, we can do another error state in case of failure

Thanks @Nirzar. I'm going to add them together in the description

"takes more than 2 seconds" - what does this refer to ? 2g connection? wifi?

@Nirzar ^ would you please answer that question?

As I understand it, though, that means that if the server hasn't responded within 2 seconds since the user's query was sent, initiate the spinner.

@Jdlrobson yes. it's 2 client seconds. irrespective of internet speed

The task is on hold until I and @Jdlrobson have a conversation as was suggested by Jon.

@bmansurov and I talked about this and we identified 5 patches:

1. Error message when no languages found in search
2. LanguageSearchGateway and unit tests
3. Async behaviour in LanguageOverlay (use LanguageSearchGateway to do search)
4. Behaviour to show "Failed to look up languages"
5. Add EventLogging for errors

@dr0ptp4kt it's not clear how https://meta.wikimedia.org/wiki/Schema:MobileWebLanguageSwitcher should incorporate errors in search. I guess this will involve adding a new "error" to the enum "event" property and a new optional errorMsg property? Can you clarify. Patch #5 is blocked on this.

It's also not clear if this should be beta only to start with. Given the risk of changing behaviour so soon to deployment (we are deploying the language switcher this sprint), beta seems to be the correct place for this but this should be added to description. Patch #3 is blocked on this. Thanks.

@Jdlrobson will it possible to test this with let's say more than 100 languages? i want to see how the search behaves on slower networks. and see all the cases in action. on our test environment we have only 5-10 languages.

@Nirzar we have more than 5-10 languages on this page https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=Lang&mobileaction=toggle_view_mobile#/languages

@Nirzar I'm not sure I follow your intentions.

The number of languages in the list will not change.
https://en.m.wikipedia.org/wiki/Barack_Obama#/languages
The only thing that introduces a delay here is how fast the search API is. You can test that in production right now: https://en.wikipedia.org/w/api.php?action=languagesearch&format=json&search=aleman - that takes 626ms even on gprs.

There are a few things I don't think we have thought through well:

1. If I search for in e.g. indonesian then English is now a valid search result as in Uzbek English is "ingliz". However the sorting is still done based on the names in the current language so English will appear above Indonesian despite Ingliz being after Indoneziya in Uzbek.
2. The async behaviour will cause a re-render of results. It's hard to tell how jarring that might be. For instance if I search for "in", as in the above example, indonesian is the 3rd result in the synchronous search. If we hit the API after 600ms English and who knows how many other languages will magically appear above.

I'm actually quite concerned this will make search less useful for some as now they have to type more to filter, so I would strongly recommend doing this in beta first (or even better prototyping) before pushing this out in front of people.

In T137680#2520034, @Jdlrobson wrote:

...
@dr0ptp4kt it's not clear how https://meta.wikimedia.org/wiki/Schema:MobileWebLanguageSwitcher should incorporate errors in search. I guess this will involve adding a new "error" to the enum "event" property and a new optional errorMsg property? Can you clarify. Patch #5 is blocked on this.

I apologize, that acceptance criterion escaped my attention. I believe what @phuedx was going for here (see https://phabricator.wikimedia.org/transactions/detail/PHID-XACT-TASK-mixbliobc4xx5up/) is that there's a non-zero chance the languagesearch endpoint will return application layer failures or may timeout or that sort of thing, and it's important to have awareness for this. Accordingly, yes, please do that very thing you suggest with "error" as a potential action and "errorState" to contain the error source; as we realized with Popups, we shouldn't log an error if our JavaScript intentionally cancelled an API call (e.g., user was typing search fast), but rather when "real" errors occur.

It's also not clear if this should be beta only to start with. Given the risk of changing behaviour so soon to deployment (we are deploying the language switcher this sprint), beta seems to be the correct place for this but this should be added to description. Patch #3 is blocked on this. Thanks.

I agree with your risk assessment. Please do this in beta first; the code should be structured as much as possible to make it easy to move it from beta to stable, naturally. Would you please also open a beta-to-stable follow up task with the appropriate level of detail? I know sometimes transition from beta to stable is more than meets the eye even if we're trying to make it simple up front to do it.

I updated the task Description to refer to beta.

@dr0ptp4kt should we abandon this ticket?