Page MenuHomePhabricator

Search Dashboard: update for engagement - sister projects
Closed, ResolvedPublic6 Story Points

Description

Based on the upcoming release of sister project snippets (T162276) in the search results pages, we'll want to track this information on the Search dashboard.

This can be added in as part of the engagement metrics - to show which projects are clicked on vs shown.

This will also require a revisiting of our earlier event logging code (T149806) to be sure it's still in place and is still what we want to capture.

Details

Related Gerrit Patches:
wikimedia/discovery/golden : master[WIP] Add sister search engagement metrics
wikimedia/discovery/golden : masterFix sister search traffic query
wikimedia/discovery/rainbow : developSister search traffic changes per Deb's feedback
wikimedia/discovery/rainbow : develop[Sister Search] Add no split option
wikimedia/discovery/rainbow : developAdd sister search traffic
wikimedia/discovery/golden : masterAdd traffic from sister search
mediawiki/extensions/WikimediaEvents : wmf/1.30.0-wmf.6Switch to data-attribute for sister-search sidebar results
mediawiki/extensions/WikimediaEvents : masterSwitch to data-attribute for sister-search sidebar results
mediawiki/extensions/WikimediaEvents : masterUpdate results tracking on SERPs

Event Timeline

debt created this task.May 9 2017, 5:16 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 9 2017, 5:16 PM

Change 352866 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[mediawiki/extensions/WikimediaEvents@master] Update results tracking on SERPs

https://gerrit.wikimedia.org/r/352866

debt added a comment.May 25 2017, 4:26 PM

As a reminder, we plan on going live with the sister project snippets the week of May 29th. The breakdown will be as follows:

The following sister project snippets will be displayed on all Wikipedias:

  • Wikisource
  • Wiktionary
  • Wikiquote
  • Wikibooks
  • Wikivoyage
  • Commons / multimedia
  • Wikinews
  • Wikiversity

The following sister project snippets will not display on enwiki:

  • Commons / multimedia
  • Wikinews
  • Wikiversity

Change 352866 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Update results tracking on SERPs

https://gerrit.wikimedia.org/r/352866

Waiting for the sister project snippets to go live, hopefully the week of June 5, 2017

mpopov added a comment.Jun 8 2017, 8:28 PM

Update: apparently the feature will go live on the train next week. Once it's live, I'll check to see if we're collecting the data we expect to be collecting.

debt added a comment.Jun 16 2017, 5:29 PM

This feature is now live in production on all wikis, please activate the dashboard! :)

Unfortunately the UI code changed in a way to invalidated my event logging patch :( the SERP code will need to be patched to include data that had been removed.

Change 359627 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[mediawiki/extensions/WikimediaEvents@master] Switch to data-attribute for sister-search sidebar results

https://gerrit.wikimedia.org/r/359627

Change 359627 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Switch to data-attribute for sister-search sidebar results

https://gerrit.wikimedia.org/r/359627

Change 360879 had a related patch set uploaded (by EBernhardson; owner: Bearloga):
[mediawiki/extensions/WikimediaEvents@wmf/1.30.0-wmf.6] Switch to data-attribute for sister-search sidebar results

https://gerrit.wikimedia.org/r/360879

Change 360879 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@wmf/1.30.0-wmf.6] Switch to data-attribute for sister-search sidebar results

https://gerrit.wikimedia.org/r/360879

Mentioned in SAL (#wikimedia-operations) [2017-06-22T18:39:18Z] <thcipriani@tin> Synchronized php-1.30.0-wmf.6/extensions/WikimediaEvents/modules/ext.wikimediaEvents.searchSatisfaction.js: SWAT: [[gerrit:360879|Switch to data-attribute for sister-search sidebar results]] T164854 (duration: 00m 50s)

While we're waiting for event logging stuff to be sorted, I'm going to add actual traffic (which uses webrequest data) as a thing.

Change 361592 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[wikimedia/discovery/golden@master] Add traffic from sister search

https://gerrit.wikimedia.org/r/361592

Change 361592 merged by Chelsyx:
[wikimedia/discovery/golden@master] Add traffic from sister search

https://gerrit.wikimedia.org/r/361592

mpopov claimed this task.Jun 27 2017, 10:23 PM
mpopov set the point value for this task to 6.

Whoops, forgot to claim this one.

Change 361902 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[wikimedia/discovery/rainbow@develop] Add sister search traffic

https://gerrit.wikimedia.org/r/361902

Change 361902 merged by Chelsyx:
[wikimedia/discovery/rainbow@develop] Add sister search traffic

https://gerrit.wikimedia.org/r/361902

@debt: let me know if there's anything you want different while we're still in the development stage or if https://discovery-beta.wmflabs.org/metrics/#sister_search_traffic is good for production :)

debt added a comment.Jun 29 2017, 3:05 PM

@mpopov - can you update for these:

  1. change ticket to T162276 in the comments
  2. update title from "Sister search traffic" to "Sister project search results traffic"
  3. update summary from

"Sister (cross-wiki) search is a feature that adds results from other projects to a sidebar on the search engine results page (SERP). For example: if there are additional results found, users are shown images from Wikimedia Commons, definitions from Wiktionary, and results from works on Wikisource."

to

"Sister project (cross-wiki) snippets is a feature that adds search results from sister projects of Wikipedia to a sidebar on the search engine results page (SERP). If a query results in matches from the sister projects, users will be shown snippets from Wiktionary, Wikisource, Wikiquote and/or other projects."

  1. move the 'A' information to under 'Notes' (not under outages)
  2. add under 'inaccuracies' that Wikisource had a unknown spike on June 22 that slightly skews that project's results and overall results for that day.
  3. can the desktop vs mobile stats be broken down a bit further by combining enwiki vs all other languages as compared to desktop and mobile stats? ie: desktop (enwiki vs all other languages and mobile (enwiki vs all other languages).
  4. same with "Search results pages vs Articles" - being able to combine this stat with languages is valuable.
  5. same with Commons/multimedia

Note: instead of radio buttons, can the selections for the display in the graph be checkboxes so the user can combine certain data sets?

  1. explain in a note that while viewing 'project' stats it includes all languages and any click into a sister project snippet (article or more results).
  2. can the stats be listed (below the graph) in a highest to lowest value for each graph selection? right now, simple wiki is always first and it's nearly always the last project in usage.
  3. add in a note that demonstrates that enwiki has a different display than all other languages and reference T162276#3278689

Thanks!

Change 363365 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[wikimedia/discovery/rainbow@develop] [Sister Search] Add no split option

https://gerrit.wikimedia.org/r/363365

Change 363365 merged by Bearloga:
[wikimedia/discovery/rainbow@develop] [Sister Search] Add no split option

https://gerrit.wikimedia.org/r/363365

Change 363510 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[wikimedia/discovery/golden@master] [WIP] Add sister search engagement metrics

https://gerrit.wikimedia.org/r/363510

Change 363752 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[wikimedia/discovery/rainbow@develop] Sister search traffic changes per Deb's feedback

https://gerrit.wikimedia.org/r/363752

Change 363752 merged by Chelsyx:
[wikimedia/discovery/rainbow@develop] Sister search traffic changes per Deb's feedback

https://gerrit.wikimedia.org/r/363752

debt added a comment.Jul 11 2017, 8:08 PM

Looks good, thanks! :)

@debt: so…are we going ahead with the idea to add another language category for languages that already have a sister project search sidebar (e.g. French)?

Do you have the list of languages that already have sister search in addition to ours?

debt added a subscriber: Jdrewniak.Jul 12 2017, 5:14 PM

Hi @mpopov - here's the list of wikis that have their own sidebar treatment of sister projects that we'll need to break out to be sure that we're not counting clicks into them as part of our sister projects:

"afwiki",
"alswiki",
"arwiki",
"barwiki",
"bgwiki",
"bxrwiki",
"cawiki",
"cawiktionary",
"cdowiki",
"cewiki",
"commonswiki",
"cswiki",
"cswiktionary",
"dawiki",
"dewiki",
"dewiktionary",
"elwiki",
"elwiktionary",
"enwiki",
"enwikibooks",
"enwiktionary",
"eowiki",
"eswiki",
"eswiktionary",
"etwiktionary",
"fawikinews",
"fawikivoyage",
"fawiktionary",
"fiwiktionary",
"fowiktionary",
"frwiki",
"frwiktionary",
"gawiktionary",
"gnwiki",
"hewikibooks",
"hewikinews",
"hewikiquote",
"hewikisource",
"huwiki",
"hywiktionary",
"iowiktionary",
"iswiki",
"iswiktionary",
"itwiki",
"jvwiki",
"kawiki",
"kawiktionary",
"kkwiki",
"kowiki",
"kowiktionary",
"kuwiktionary",
"lowiki",
"lvwiki",
"mgwiktionary",
"mkwiki",
"mkwiktionary",
"mlwiki",
"mlwiktionary",
"mrwiki",
"nds_nlwiki",
"ndswiki",
"newiki",
"newikibooks",
"newiktionary",
"nlwiki",
"nowiki",
"nrmwiki",
"nvwiki",
"pawiki",
"pawiktionary",
"plwiki",
"plwikinews",
"plwiktionary",
"pswiki",
"ptwiki",
"ptwikibooks",
"rowiki",
"rowiktionary",
"ruwiki"
"ruwikiquote",
"ruwikisource",
"ruwiktionary",
"simplewiktionary",
"sqwiki",
"svwiki",
"testwikidatawiki",
"thwiki",
"tkwiktionary",
"trwiki",
"trwiktionary",
"ugwiktionary",
"ukwiki",
"urwiki",
"viwiki",
"viwikibooks",
"viwikisource",
"viwiktionary",
"wikidatawiki",
"zh_min_nanwiki",
"zhwiki",
"zh_yuewiki",

Change 365190 had a related patch set uploaded (by Bearloga; owner: Bearloga):
[wikimedia/discovery/golden@master] Fix sister search traffic query

https://gerrit.wikimedia.org/r/365190

Change 365190 merged by Chelsyx:
[wikimedia/discovery/golden@master] Fix sister search traffic query

https://gerrit.wikimedia.org/r/365190

debt added a comment.Jul 17 2017, 10:05 PM

Here's a list of all the Wikipedias that are using the “Sisterproject” or some other message override to display links from Wikidata and other projects, based on an automated analysis of every search result page done by @Jdrewniak :

az.wiki
cdo.wiki
cs.wiki
da.wiki
dsb.wiki
el.wiki
eo.wiki
es.wiki
eu.wiki
sc.wiki
lb.wiki
sw.wiki
gu.wiki
he.wiki
hsb.wiki
id.wiki
ilo.wiki
it.wiki
kk.wiki
ky.wiki
la.wiki
pdc.wiki
pl.wiki
ro.wiki
sa.wiki
sh.wiki
sk.wiki
szl.wiki
th.wiki
tl.wiki
udm.wiki
vi.wiki
or.wiki

gh87 added a subscriber: gh87.Jul 28 2017, 2:39 AM
debt closed this task as Resolved.Aug 30 2017, 9:59 PM
debt moved this task from Backlog to Done on the Discovery-Analysis (Current work) board.

It looks like we've done enough for determining the engagement of the sister project snippets at this time, so closing this ticket.

Change 363510 abandoned by Bearloga:
[WIP] Add sister search engagement metrics

https://gerrit.wikimedia.org/r/363510