Here are the updated numbers:

@Elitre , do you know if T276257 - "Create "assessment" filter in MediaSearch" - satisfies this use case?

In T273092#7050841, @PPham wrote:

And so I really want to know how many are left with the Vietnamese language...

@gmodena can you say more about why we're filtering by label language at all? Don't we want to extract labels regardless of language?

The latest update, as of 4:27pm UTC:

In T273092#7042316, @Ankan_WMF wrote:

Some articles' contents can be culturally different for the user who will be using the image recommendation tool. In that case, the user needs to click on the image, and check whether it's a match or not by reading the description or checking the articles to which the image is already linked. Some users may understand due to their background knowledge, but it will not be the usual case. For these cases, is it ok to mark them as "unsure"?

In T260254#6930808, @nettrom_WMF wrote:

In T258229#6981007, @nettrom_WMF wrote:

We now have a draft dashboard on Superset that covers almost all the metrics specified in this task. Statistics are gathered daily and are driven by two notebooks that can be found on GitHub: this one aggregates the legacy and MediaSearch statistics, and this one aggregates the MediaSearch-specific funnel metrics. The dashboard then pulls in the data and calculates any necessary percentages on the fly.

I propose that we hold off on calculating how many filters and what filters are used until we have a few weeks of usage data, rather than aggregating that on a daily basis.

Given that MediaSearch was first made default for non-logged-in users and will become default for logged-in users later, I suspect we'll be wanting to be able to split statistics by whether the user was logged in or not. That's currently not possible as the mediasearch_interaction schema doesn't capture that. If we need that as well, we'll need two additional tasks, one to update the schema and instrumentation of Special:MediaSearch, and one to update the aggregation.

I'll discuss all of this with @CBogen in our next sync meeting and create tasks depending on what the needs are.

In T273092#7034894, @Dyolf77_WMF wrote:

In T273092#7033965, @CBogen wrote:

When you say duplicates, what do you mean? Duplicate articles? Duplicate images? Are you seeing these duplicates after already marking them as good or bad matches?

Thanks, I mean the tool is showing these cases of suggestions:

• the same image with the same article more than once (I marked the image with Okay),
• the same image with different articles (Image marked bad),
• the same article with different image suggestions (Images marked bad/good).

In T280055#7030660, @egardner wrote:

Does the generator-based search that we're using support searching special pages?

I just tried to query the API directly using the sandbox on commons; I supplied the relevant params that Special:MediaSearch uses (generator=search, gsrnamespace=-1, etc).

Despite providing only -1 as the namespace value (the NS for special pages), I don't see any special pages in the results. Instead, the response includes the following warning:

"warnings": {
    "search": {
        "*": "Unrecognized value for parameter \"gsrnamespace\": -1"
    }
}

In T280370#7035000, @Cparle wrote:

This will take quite a while to run. There are about 13M unillustrated articles across all languages, and looking at the API response times for populating the data for https://image-recommendation-test.toolforge.org/ it takes approx 0.1s to get one unillustrated-article-plus-recommendations. That means 1.3 million seconds to get them all, which is around 15 days ... and that's just to get the articles, without looking into users and their languages etc

While there's some work going on around improving response times I'm going to fake the image recommendation API and work on the rest of the the stuff

In T280425#7030642, @egardner wrote:

I should note that the open patch here does not exactly align with the acceptance criteria. What happens instead is that users who arrive on the page with an invalid type parameter (?type=bitmap, ?type=foo, etc) will instantly and silently get redirected to (?type=image), the default search type. All other params remain and the UI then loads normally. The URL gets updated to include image as well, so users copying-and-pasting the resulting page will have valid links going forward.

If this behavior is not adequate, we may need to end up re-working how we do error handling here. Right now we are not doing server-side error handling at all.

In T273092#7031293, @Dyolf77_WMF wrote:

@CBogen Hi, I noticed that the tool is displaying many duplicates. And sometimes in image descriptions (in Wikimedia Commons) there are words that match with the article, but the image itself is not at all in the scope of the article.

Note: Once we have a confidence score, we'll want to decide what the cutoff confidence score should be for image recommendations (and does it differ by use case?) Also, we'll want to measure how much cutting off by each confidence score decreases the coverage of image recommendations matches returned by MediaSearch.

Now blocked by T273092

Name of approving party (manager for WMF/WMDE staff): @CBogen

In T273092#7024335, @CBogen wrote:

@Urbanecm_WMF @Dyolf77_WMF @PPham @Ankan_WMF I spoke too soon, there's another issue. Please pause testing. My apologies. I'll update shortly.

@Urbanecm_WMF @Dyolf77_WMF @PPham @Ankan_WMF I spoke too soon, there's another issue. Please pause testing. My apologies. I'll update shortly.

@Urbanecm_WMF @Dyolf77_WMF @PPham @Ankan_WMF We are ready to go! The tool is here:

There's been another delay due to some bugs and now we're looking at Wednesday April 21st (hopefully) or definitely by Thursday the 22nd.

There’s been a short delay and we now expect to be ready to start evaluation on Tuesday April 20th. Thanks for your patience all!

Unfortunately updating from the dump takes time and we can't initiate it over the weekend. We are working this quarter (see T260568) on adding streaming updates to WCQS so this hopefully won't be an issue in the future.

@matthiasmullie @Cparle The search team is trying to understand why SD chose to build their own tools instead of using the existing tools. They want to own one set of tooling only, and so they need to understand what was missing from the previously existing tools, so they know how to improve (or whether to replace) the existing tools. Can you provide some input? Thanks!

Some information from the folks at Mozilla: