Verify data pipeline for TextCat A/B test on English Wikipedia
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Deskana
	May 3 2016, 9:45 PM

Description

After the test is deployed (see T121542), we should verify the data pipeline and data collection for the test.

Related Objects
Search...

Status	Assigned	Task
Open	None	T118278 [EPIC] Improve Language Identification for use in Cirrus Search
Resolved	EBernhardson	T121543 Do an A/B Tests on Other Wikis with TextCat for Language Identification
Resolved	EBernhardson	T137158 Compile and then resolve issues with TextCat A/B test data
Declined	mpopov	T134320 Analyse results of TextCat A/B test
Resolved	EBernhardson	T130321 Disable Schema:Search, since it's outdated and redundant
Resolved	mpopov	T129564 Switch Desktop data collection for dashboards to use TestSearchSatisfaction2 instead of Search schema
Resolved	EBernhardson	T134319 Turn off TextCat A/B test on the English Wikipedia on or after May 23
Resolved	debt	T134318 Verify data pipeline for TextCat A/B test on English Wikipedia
Resolved	EBernhardson	T121542 Write and deploy an A/B Test on enwiki using TextCat for Language Identification
Resolved	dcausse	T121540 Investigate Updating Cybozu / ES Plugin for Language Identification
Resolved	EBernhardson	T124844 Add textcat to mediawiki vendor libs
Resolved	Smalyshev	T121538 Convert TextCat to PHP Library for Language Identification in Cirrus Search
Resolved	TJones	T123537 Generate wikitext-based and query-based language models for TextCat
Resolved	TJones	T123651 Decide which set of separators we have to use for TextCat ngrams
Resolved	• dpatrick	T123558 Security review for TextCat library
Resolved	mpopov	T132706 Validate click events in TestSearchSatisfaction2

Event Timeline

• Deskana created this task.May 3 2016, 9:45 PM

Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptMay 3 2016, 9:45 PM

• Deskana mentioned this in T134319: Turn off TextCat A/B test on the English Wikipedia on or after May 23.May 3 2016, 9:46 PM

• Deskana triaged this task as Medium priority.May 3 2016, 9:49 PM

• Deskana added a parent task: T134319: Turn off TextCat A/B test on the English Wikipedia on or after May 23.

• Deskana added a subtask: T121542: Write and deploy an A/B Test on enwiki using TextCat for Language Identification.

• Deskana moved this task from Needs triage to Up Next on the Discovery-Analysis board.May 5 2016, 8:05 PM

It's been a week, so according to the blocked task it's time to turn the test off. Is the data usable?

Well, crap. I created this task and didn't prioritise it properly. This verification is happening late. I'm increasing priority to try to rectify this situation.

mpopov moved this task from Backlog to In progress on the Discovery-Analysis (Current work) board.May 24 2016, 11:28 PM

I checked out the data and saw a few issues that have been addressed by Erik in this etherpad: https://etherpad.wikimedia.org/p/r.a64562df192c43b04c4fbe57d439ded4

I now have a script for fetching and correctly processing the TextCat A/B test EL data. (See https://github.com/wikimedia-research/Discovery-Search-Test-TextCat/blob/master/data.R)

Looks like we're good to turn the test off, @EBernhardson

mpopov moved this task from In progress to Done on the Discovery-Analysis (Current work) board.May 25 2016, 10:00 PM

mpopov mentioned this in T137158: Compile and then resolve issues with TextCat A/B test data.Jun 6 2016, 10:16 PM

debt closed subtask T121542: Write and deploy an A/B Test on enwiki using TextCat for Language Identification as Resolved.Jun 8 2016, 12:37 AM

Looks like this is resolved - closing.

debt moved this task from Done to Resolved on the Discovery-Analysis (Current work) board.Jul 20 2016, 4:23 PM

Verify data pipeline for TextCat A/B test on English WikipediaClosed, ResolvedPublicActions

Description

Related ObjectsSearch...

Event Timeline

Verify data pipeline for TextCat A/B test on English Wikipedia
Closed, ResolvedPublic
Actions

Related Objects
Search...