In the heatmap I was surprised to see better examination probablities at pos+1 for a given prefix, but reading the paper I note (1f contraint) :

This is fixed after merging the patch from Stas. As per @EBernhardson investigations it appears to be due to some dirty data left in the cache by the test itself. While this fixed this particular problem I'm not yet sure to fully understand what went wrong here especially why the test suite did pass properly when running CI on CirrusSearch but not on other extensions...

Analyzing few queries from commons and manually tuning relevance signals using relforge tools for the first time was very insightful. The goal was to try to evaluate the benefit of adding a query independent signal based on aesthetic quality computed by a tensorflow model provided by Miriam Redi.

Since the query sent to elastic hugely varies depending on the user language we should keep this information alongside the query text or we will completely skew the data-set.

In T205111#4611327, @EBernhardson wrote:

• Basic stats about distribution of the length of prefix typed / number of characters saved.

It was probably due to inconsistencies that are now fixed automatically by the Sanitizer (ref T139200).

technical note:

• on the text field we split on camel case
• on the plain field we do not split

Fixed, sorry about that and thanks for the @Cparle!

Should be fixed by https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/461622

I think using SiteInfo might be the easiest solution, we could add wmgCirrusSearchDefaultCluster besides wmfMasterDatacenter and warn if it's not local?

@Miriam thanks for your suggestions!

• Analyzing the performance vs our click data will require running your model against many more images and I wonder if it'd not be simpler to simply run the model against all images on commons.
• Qualitatively: thanks, we have been looking at rescoring the top-8000 by mixing the relevance and the quality score, we can try to rescore only the top-10. I'll look into setting such profile.

Using the formula suggested by Erik (a boosting factor 1 + 0.25 * (score-0.5)) I get fewer differences:

Metrics:
   Query Count: 90
   Zero Results Rate: 0.0%
   Poorly Performing Percentage: 0.0%
   Top 1 Unsorted Results Differ: 42.2%
   Top 3 Sorted Results Differ: 87.8%
   Top 3 Unsorted Results Differ: 73.3%
   Top 5 Sorted Results Differ: 93.3%
   Top 5 Unsorted Results Differ: 77.8%
   Top 20 Sorted Results Differ: 98.9%
   Top 20 Unsorted Results Differ: 87.8%

Stats comparing all field * templates_boost vs all field + 0.6*quality score (on a subset of commons)

Prefixes that match pass by are:

• Pass By:
  • Pass By Name => Evaluation_strategy#Call_by_name
  • Pass By Reference => Evaluation_strategy#Call_by_reference
  • Pass By Value => Evaluation_strategy#Call_by_value
• Pass by:
  • Pass by catastrophe
  • Pass by reference => Evaluation_strategy#Call_by_reference
  • Pass by value => Evaluation_strategy#Call_by_value
• Pass-By
  • Pass-By-Name => Evaluation_strategy#Call_by_name
  • Pass-By-Name Evaluation => Evaluation_strategy#Call_by_name
  • Pass-By-Reference => Evaluation_strategy#Call_by_reference
  • Pass-By-Reference Evaluation => Evaluation_strategy#Call_by_reference
  • Pass-By-Value =>Evaluation_strategy#Call_by_value
  • Pass-By-Value Evaluation =>Evaluation_strategy#Call_by_value
• Pass-by
  • Pass-by-reference => Evaluation_strategy#Call_by_reference
  • Pass-by-value => Evaluation_strategy#Call_by_value

@Miriam: evaluation is done by us for now, it's just the very first step to adjust the knobs to something reasonable. If we manage to find a good balance between the quality score and the relevance score we should discuss the next steps.
Thanks!

Size is already taken into account by the scoring mechanism (see text_word_count in P7481) unfortunately this does not help enough to move Thierry to the top.
I don't see a particular feature we don't use yet use to push this page to the top.
Perhaps football/soccer/american football ambiguity is part of the problem, Thierry is ranked #2 for henry soccer.
Thanks for reporting this but not sure we can do much for this particular query.

[redundant comment]

If this ticket is about matching (without using any kind of search keyword) an entity referencing a string regardless of its usage (label/alias/statements) then why not simply put all the statement strings into the text field or auxilliary_text (it's not used currently)?
To save some space we could also stop populating the source_text field for entities I doubt that insource is helpful on wikidata.

Indeed this is very similar and most probably I should close this one as a duplicate. Since we recently worked on pagination I thought we introduced this behavior but looks like this bug was already here.

this will happen in cirrus, not as an elasticsearch plugin

As pointed out by Manybubbles this will be completely fixed once we fully get rid of elasticsearch query_string. This work is being tracked in T185108.

moving to waiting/blocked waiting for the hebrew plugin

If this submodule is not referenced by anything else other than mediawiki then there should be no reason to bump it in deployment-prep, I'd be for resetting it to 4db9d40d28d61c53cdbca77059d9a2a6e714af89 to match prod.
Side note: I'm pretty sure that https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/300880 was unnecessary. This submodule in mw-config is not meant to be updated whenever a change to it is applied, only when schema used by MW is updated.
Then to the question: why event-schema was bumped directly on deployment-tin? I have no clue, so if nobody requires it I'd be for cleaning it and align it to prod.

thanks!
this works as expected now.

Pinging @Pchelolo as he may use this repo for testing purposes.