This should be blocked on fixing T95026: PageImages should be able to pick Wikidata's P18 as chosen image.

This should be blocked on fixing T95026: PageImages should be able to pick Wikidata's P18 as chosen image.

We'll use the standardized approach - using the PageImages API - that we're using in other places. It would be great if T95026 would be fully fixed, but I don't think it's a blocker to this A/B test.

I'm now wondering: will the images be clickable? (In theory they should be, to give attribution.) Images are click-traps, so I expect it will be useful to know whether they "steal" some clicks from actual search results.

In short: please count clicks on images and clicks on pages, but separately.

I also wonder how to measure and assess whether the images make people click different search results: the screenshot shows the first three results as having an image, but I guess it could also happen that the first result has no image and the following three have. I'd expect users to become more likely to click the non-first results that they would otherwise. But then this doesn't tell us whether the image-endowed result was actually better or worse for them.

Maybe recording the number of images displayed and the position of the clicked results (which you already do or plan to do, AFAICT) will be enough for later analysis if needed.

@Nemo_bis In this context, thumbnails serve just as a visual cue for the search result, and don't have any intrinsic value on their own. This is why on mobile search (typeahead) and on other search engines, clicking on the image acts just like clicking on the text. The thumbnail + search result are perceived as one unit, so it would be confusing if the text led you to one page but the image to another.

In the code I've written so far for this test, I've made the image not clickable (mostly for simplicity), but if they were clickable, I think it would make the most sense to make them lead to the search result page as well. Given that the page-images are probably the most prominent image on an article page, I'd leave it to users to click through to the image from the there if they are interested in it.

recording the number of images displayed and the position of the clicked results

Yes, this is what we plan to do. we'll record the number of images displayed, and their position, as well position of the clicked result (which we do already).

A demo of this feature is available for testing by adding the following line to a users common.js file on english wikipedia:
importScript( 'User:JDrewniak_(WMF)/searchThumbs.js' );

Cool demo Jan. Now I want the image to link to the article. :)

@mpopov regarding the event logging for this test, I'd like to go over some details of how information about these thumbnails can be collected.
I have an idea that I think integrates nicely with the existing SearchSatisfaction schema.

Currently, when a user is selected for a test group, a searchResultPage event is logged when the search page is loaded. This event already contains information about the results presented within the extraParams property (number & order of sister search results, pagination offset, etc). For this test, I propose we add a value to extraParams for the searchResultPage event named something like thumbnailPositions that would contain an array of the positions of the thumbnails, so [3, 5, 10] would mean thumbnails were presented for the 3rd 5th 10th search results. The length of the array would give us the total number of thumbnails presents, and when a user clicks through to a result, since we already log which position was clicked, we can just correlate whether that position had a thumbnail or not.

Does that seem like a sound idea?

In T149811#3530016, @Nemo_bis wrote:

I'm now wondering: will the images be clickable? (In theory they should be, to give attribution.)

Not necessarily.

If a free image is used in a thumbnail on a search result, then the image licensing for that image is two clicks away from where the image is displayed, i.e. click on the result to go to article, then click on the image in the article. This is well within the typically accepted bounds of sufficient attribution for most (if not all?) free licences. For fair use images, attribution technically isn't required at all, so two clicks would also be acceptable. There's not a big difference between two clicks (as in the search results) and one click (as in articles), especially when it's clear that the first click will take you to the article in question which contains the image.

(I am not a lawyer, I'm just someone who's dealt with this exact problem before, i.e. what is sufficient attribution for images and text in terms of how many clicks away it is from another page.)

In T149811#3533105, @Jdrewniak wrote:

@mpopov regarding the event logging for this test, I'd like to go over some details of how information about these thumbnails can be collected.
I have an idea that I think integrates nicely with the existing SearchSatisfaction schema.

Currently, when a user is selected for a test group, a searchResultPage event is logged when the search page is loaded. This event already contains information about the results presented within the extraParams property (number & order of sister search results, pagination offset, etc). For this test, I propose we add a value to extraParams for the searchResultPage event named something like thumbnailPositions that would contain an array of the positions of the thumbnails, so [3, 5, 10] would mean thumbnails were presented for the 3rd 5th 10th search results. The length of the array would give us the total number of thumbnails presents, and when a user clicks through to a result, since we already log which position was clicked, we can just correlate whether that position had a thumbnail or not.

Does that seem like a sound idea?

Sounds good! And if you decide to make the images clickable after all, you can likewise include a boolean thumbnail flag in click events' extraParams :)

Change 373049 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[mediawiki/extensions/WikimediaEvents@master] [WIP] Search thumbnails AB test

https://gerrit.wikimedia.org/r/373049

Moving this to the backlog for now, as other priorities have surfaced.

If a user wants to add this into their logged-in account, simply add this script to their common.js file:

importScript( 'User:JDrewniak_(WMF)/searchThumbs.js' );

Change 373049 abandoned by Jdrewniak:
Search thumbnails AB test

Reason:
we won't be going forward with this AB test.

https://gerrit.wikimedia.org/r/373049