Post-deployment QA for language switching functionality
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	ovasileva
	Apr 21 2021, 1:20 PM

Description

Over the course of development of the new language switching functionality, we were testing only on the beta cluster. This means we have yet to do QA in production. This will be done once the feature is deployed.

QA steps

Config checks

Go to https://en.wikipedia.org/wiki/Paris as an anonymous user

Ensure no language button appears

Go to https://fr.wikipedia.org/wiki/Paris as an anonymous user

Ensure no language button appears

Go to https://fr.wikipedia.org/wiki/Paris as a logged-in user

Ensure no language button appears

Overall Functionality

Go to https://en.wikipedia.org/wiki/The_Heart_of_Thomas and ensure you are opted out of legacy vector (you are seeing the new version of the skin)

Ensure the language button appears at the top of the page
Test at various widths and note any inconsistencies
Ensure the featured article indicator appears below the line

Select language button

Ensure ULS appears directly under language button
Ensure languages appear in one column only

Go to https://en.wikipedia.org/wiki/Paris

Ensure languages appear in multiple columns

Settings

Go to https://en.wikipedia.org/wiki/Paris

Ensure the display and input setting buttons appear at the bottom and open the settings modal when selected

Empty state

Go to https://en.wikipedia.org/wiki/John_Blackman_(cricketer)

Ensure no language button appears on the page

Indicator check

Go to https://en.wikipedia.org/wiki/Paris

Ensure language button appears
Ensure all indicators appear below the line

Special pages

Go to https://en.wikipedia.org/wiki/Special:RecentChanges

Ensure no language button appears

RTL check

Go to https://ar.wikipedia.org/wiki/%D8%A8%D9%88%D8%B1%D8%B3%D8%B9%D9%8A%D8%AF

Ensure you are opted out of the legacy version of the skin (you can see new vector)
Ensure language button appears, opens and closes, and there are no visual inconsistencies

Checking Commons and Wikidata

Go to https://commons.wikimedia.org/wiki/Category:Animalia and ensure you are logged-in and seeing the latest version of the vector skin

No language selector should appear

Go to https://www.wikidata.org/wiki/Q2 and ensure you are logged-in and seeing the latest version of the vector skin

No language selector should appear

Indicator visual checks

For the wikis linked below, ensure you are logged in and can see the latest version of vector. Do a visual spot check that the language button appears and is not overlapping any coordinates or article indicators.

English Wikipedia - https://en.wikipedia.org/wiki/Kyiv
Spanish Wikipedia - https://es.wikipedia.org/wiki/Dmitri_Shostak%C3%B3vich, https://es.wikipedia.org/wiki/Mashhad
German Wikipedia - https://de.wikipedia.org/wiki/Napoleon_Bonaparte, https://de.wikipedia.org/wiki/St._Helena_(Insel)
Japanese Wikipedia - https://ja.wikipedia.org/wiki/%E6%B7%B1%E6%B5%B7%E9%AD%9A, https://ja.wikipedia.org/wiki/%E3%83%91%E3%83%AA
Russian Wikipedia - https://ru.wikipedia.org/wiki/%D0%A4%D0%B5%D0%B4%D0%BE%D0%BD_(%D0%9F%D0%BB%D0%B0%D1%82%D0%BE%D0%BD), https://ru.wikipedia.org/wiki/%D0%90%D1%84%D0%B8%D0%BD%D1%8B
Italian Wikipedia - https://it.wikipedia.org/wiki/Abruzzo, https://it.wikipedia.org/wiki/Avezzano
Chinese Wikipedia - https://zh.wikipedia.org/wiki/1880%E5%B9%B4%E7%BE%8E%E5%9B%BD%E6%80%BB%E7%BB%9F%E9%80%89%E4%B8%BE, https://zh.wikipedia.org/wiki/%E5%8C%97%E4%BA%AC%E5%B8%82
Arabic Wikipedia - https://ar.wikipedia.org/wiki/%D8%A8%D8%A7%D8%B1%D9%8A%D8%B3
Polish Wikipedia - https://pl.wikipedia.org/wiki/Jos%C3%A9_Cardona, https://pl.wikipedia.org/wiki/Honduras
Indonesian Wikipedia - https://id.wikipedia.org/wiki/Mosaik-mosaik_Delos, https://id.wikipedia.org/wiki/Daerah_Khusus_Ibukota_Jakarta
Dutch Wikipedia - https://nl.wikipedia.org/wiki/Jakarta, https://nl.wikipedia.org/wiki/Bevrijdingsdag
English Wiktionary - https://en.wiktionary.org/wiki/guddle#English
Ukrainian Wikipedia - https://uk.wikipedia.org/wiki/%D0%9A%D0%B8%D1%97%D0%B2, https://uk.wikipedia.org/wiki/%D0%93%D1%80%D0%B0_(%D1%84%D1%96%D0%BB%D1%8C%D0%BC)
Vietnamese Wikipedia - https://vi.wikipedia.org/wiki/Franz_Kafka, https://vi.wikipedia.org/wiki/Praha
Swedish Wikipedia - https://sv.wikipedia.org/wiki/Prag, https://sv.wikipedia.org/wiki/N%C3%B6dbost%C3%A4der_i_Stockholm
Czech Wikipedia - https://cs.wikipedia.org/wiki/Stockholm, https://cs.wikipedia.org/wiki/Operace_Izotop
Thai Wikipedia - https://th.wikipedia.org/wiki/%E0%B8%99%E0%B8%B9%E0%B8%81%E0%B8%B9%E0%B8%AD%E0%B8%B2%E0%B9%82%E0%B8%A5%E0%B8%9F%E0%B8%B2
Hungarian Wikipedia - https://hu.wikipedia.org/wiki/Nuku%CA%BBalofa

QA Results - Prod

✅ AC1: Config Checks

1.1	✅	T280788#7067232
1.2	✅	T280788#7067232
1.3	✅	T280788#7067232

❌ AC2: Overall Functionality

2.1	✅	T280788#7067232
2.2	✅	T280788#7067232
2.3	✅	T280788#7067232
2.4	✅	T280788#7067232
2.5	✅	T280788#7067232
2.6	❌	T280788#7067232

✅ AC3: Settings

3.1

✅

T280788#7067232

✅ AC4: Empty State

4.1

✅

T280788#7067232

✅ AC5: Indicator Check

5.1	✅	T280788#7067232
5.2	✅	T280788#7067232

✅ AC6: Special Pages

6.1

✅

T280788#7067232

❌ AC7: RTL Check

7.1	✅	T280788#7067232
7.2	❌	T280788#7067232

❌ AC8: Checking Commons and Wikidata

8.1	❌	T280788#7067232
8.2	✅	T280788#7067232

❌ AC9: Indicator visual checks

9.1	✅	T280788#7067232
9.2	❌	T280788#7067232
9.3	❓	T280788#7067232
9.4	✅	T280788#7067232
9.5	✅	T280788#7067232
9.6	✅	T280788#7067232
9.7	✅	T280788#7067232
9.8	❌	T280788#7067232
9.9	✅	T280788#7067232
9.10	✅	T280788#7067232
9.11	✅	T280788#7067232
9.12	✅	T280788#7067232
9.13	✅	T280788#7067232
9.14	✅	T280788#7067232
9.15	✅	T280788#7067232
9.16	✅	T280788#7067232
9.17	✅	T280788#7067232
9.18	❌	T280788#7067232
9.19	❌	T280788#7067232

Related Objects
Search...

Status	Assigned	Task
Resolved	ovasileva	T275766 Log sessionID, isAnon, editBucketCount, and selectedLanguage properties in the UniversalLanguageSelector instrument
Resolved	ovasileva	T275794 Log time on page before switching language as part of the UniversalLanguageSelector instrument
Resolved	ovasileva	T256023 [EPIC] Improve language switching capabilities
Resolved	ovasileva	T269093 Deploy new language switching location to test wikis and begin A/B test pt 1
Resolved	ovasileva	T280788 Post-deployment QA for language switching functionality
Resolved	Jdrewniak	T280526 Deploy new language switching functionality to logged-in users
Resolved	ovasileva	T277588 It should be possible to enable the new language button for logged in users
Resolved	phuedx	T275762 Instrument clicks to links in the Languages list in the sidebar
Resolved	ovasileva	T268504 [SPIKE 24hrs] Instrument language button
Resolved	ovasileva	T276248 Position language menu below language button

Event Timeline

ovasileva created this task.Apr 21 2021, 1:20 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 21 2021, 1:20 PM

ovasileva triaged this task as High priority.Apr 21 2021, 1:20 PM

ovasileva added a subtask: T280526: Deploy new language switching functionality to logged-in users .

ovasileva added a parent task: T269093: Deploy new language switching location to test wikis and begin A/B test pt 1.

Jdlrobson moved this task from Needs Analysis to QA on the Web-Team-Backlog (Kanbanana-FY-2020-21) board.Apr 26 2021, 9:54 PM

In interest of keeping all the bugs from QA in one place: @Nikerabbit found this one: T280526#7040745
escaping issue (32 langues) that is visible in French: https://test.wikipedia.org/wiki/Main_Page?uselang=fr

Not really sure why that's happening (but can replicate locally)

ovasileva moved this task from QA to QA in Prod on the Web-Team-Backlog (Kanbanana-FY-2020-21) board.May 3 2021, 5:58 PM

ovasileva updated the task description. (Show Details)May 5 2021, 10:55 AM

Restricted Application added subscribers: Stang, Base. · View Herald TranscriptMay 5 2021, 10:55 AM

Test Result - Prod

Status: ❌ Fail
Environment: refer to step urls
OS: macOS Big Sur
Browser: Chrome
Device: MBP
Emulated Device: NA

Test Artifact(s):

QA steps

✅ AC1: Config checks

Go to https://en.wikipedia.org/wiki/Paris as an anonymous user
✅ AC1.1: Ensure no language button appears

Screen Shot 2021-05-06 at 5.37.49 AM.png (1×1 px, 996 KB)

Go to https://fr.wikipedia.org/wiki/Paris as an anonymous user
✅ AC1.2: Ensure no language button appears

Screen Shot 2021-05-06 at 5.40.30 AM.png (1×1 px, 983 KB)

Go to https://fr.wikipedia.org/wiki/Paris as a logged-in user
✅ AC1.3: Ensure no language button appears

Screen Shot 2021-05-06 at 5.41.53 AM.png (1×1 px, 933 KB)

❌ AC2: Overall Functionality

Go to https://en.wikipedia.org/wiki/The_Heart_of_Thomas and ensure you are opted out of legacy vector (you are seeing the new version of the skin)

Screen Shot 2021-05-06 at 5.52.07 AM.png (1×1 px, 735 KB)

✅ AC2.1: Ensure the language button appears at the top of the page

Screen Shot 2021-05-06 at 5.48.10 AM.png (1×1 px, 707 KB)

✅ AC2.2: Test at various widths and note any inconsistencies
The language button appeared consistent between 500-3440px.
✅ AC2.3: Ensure the featured article indicator appears below the line
see above.

Select language button

Screen Shot 2021-05-06 at 5.53.09 AM.png (1×1 px, 576 KB)

✅ AC2.4: Ensure ULS appears directly under language button
✅ AC2.5: Ensure languages appear in one column only

Go to https://en.wikipedia.org/wiki/Paris
❌ AC2.6: Ensure languages appear in multiple columns
If I navigate directly to the page by entering the URL.

Screen Shot 2021-05-06 at 5.54.20 AM.png (1×1 px, 745 KB)

If I go back to a previous page then click forward, I initially get 1 column, and sometimes it closes immediately. On the second click, I will get multiple columns. I tried this with a more realistic scenario where I go to a page that links to the Paris page and click the Paris link to navigate to the page. I get the same initial single-column result.

Screen Recording 2021-05-06 at 5.58.31 AM.mov.gif (1×1 px, 2 MB)

Screen Recording 2021-05-06 at 6.12.36 AM.mov.gif (1×1 px, 1 MB)

✅ AC3: Settings

Go to https://en.wikipedia.org/wiki/Paris
✅ AC3.1: Ensure the display and input setting buttons appear at the bottom and open the settings modal when selected

Screen Recording 2021-05-06 at 6.18.05 AM.mov.gif (1×1 px, 1 MB)

✅ AC4: Empty state

Go to https://en.wikipedia.org/wiki/John_Blackman_(cricketer)
✅ AC4.1: Ensure no language button appears on the page

Screen Shot 2021-05-06 at 6.20.46 AM.png (1×1 px, 406 KB)

✅ AC5: Indicator check

Go to https://en.wikipedia.org/wiki/Paris

Screen Shot 2021-05-06 at 6.22.56 AM.png (1×1 px, 995 KB)

✅ AC5.1: Ensure language button appears
✅ AC5.2: Ensure all indicators appear below the line

✅ AC6: Special pages

Go to https://en.wikipedia.org/wiki/Special:RecentChanges
✅ AC6.1: Ensure no language button appears

Screen Shot 2021-05-06 at 6.24.46 AM.png (1×1 px, 557 KB)

❌ AC7: RTL check

Go to https://ar.wikipedia.org/wiki/%D8%A8%D9%88%D8%B1%D8%B3%D8%B9%D9%8A%D8%AF
✅ AC7.1: Ensure you are opted out of the legacy version of the skin (you can see new vector)
❌ AC7.2: Ensure language button appears, opens and closes, and there are no visual inconsistencies
Button opens and closes but the indicator is not below the line. The spacing between the language button and the line seems too large.

Screen Recording 2021-05-06 at 6.31.36 AM.mov.gif (1×1 px, 609 KB)

Screen Shot 2021-05-06 at 6.36.33 AM.png (1×1 px, 711 KB)

❌ AC8: Checking Commons and Wikidata

Go to https://commons.wikimedia.org/wiki/Category:Animalia and ensure you are logged-in and seeing the latest version of the vector skin
❌ AC8.1: No language selector should appear

Screen Shot 2021-05-06 at 6.45.20 AM.png (1×1 px, 472 KB)

Go to https://www.wikidata.org/wiki/Q2 and ensure you are logged-in and seeing the latest version of the vector skin
✅ AC8.2: No language selector should appear