The existing common terms test is interesting, but not part of our quarterly goals. To get things moving forward we need to change the sub test over to using the new language search. For the most part this should just be changing the form parameters it is adding to have the hidden field 'cirrusAltLanguage' set to 'yes'. To determine if the result page had an interwiki response should be able to check for p.mw-search-interwiki-header
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Change search satisfaction subtest to language search | mediawiki/extensions/WikimediaEvents | master | +14 -26 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Ironholds | T118483 Update satisfaction table references | |||
| Declined | EBernhardson | T117442 Change search satisfaction schema sub test from common terms to language search |
Event Timeline
Comment Actions
@mpopov what do we need to collect for the control group here? The language detection feature only runs on queries with zero results, so flagging the queries that could have been effected but wern't isn't particularly useful.
Perhaps the control group doesn't need to be specified, and you just take a random sampling of people within the satisfaction schema?
Comment Actions
Change 250571 had a related patch set uploaded (by EBernhardson):
[WIP] Change search satisfaction subtest to language search
Comment Actions
I think what we're interested in is whether our "satisfaction" metrics for the language search folks are consistent with what we've been seeing. Like, we shouldn't expect their clickthroughs and time-on-page to be greater than controls'. What we should hope for is that those things aren't much lower than controls.
Yeah, I'll just take a random sample for the control group.
Comment Actions
Change 250571 abandoned by EBernhardson:
Change search satisfaction subtest to language search
Reason:
the generated links go to another domain, so the existing method of using localStorage in the satisfaction schema will not work for the inter-wiki use case. Will work up another patch to measure click through.
Comment Actions
After reviewing the previous test we ran, we basically already have this information just not in the event logging table. Within hive wmf.webrequest table all interwiki clicks are recorded with x_analytics_map['wprov'] = 'iwsw1'. This was already going to be a poor substitute for the default satisfaction schema, due to the inability to pass localstorage data across sub-domains. As such marking this 'declined'. We will source the appropriate data about clickthroughs from hive.