This problem can be reproduced at other wikis besides German Wikipedia (and possibly for other task types besides Add Link).

I haven't checked, but I presume this problem is caused by the fact articles and topics have an N:M relationship. If GrowthExperiments is making a separate query for each topic and summing their individual counts together, then we would be counting one article a couple of times, resulting in this confusion.

I'm curious whether this impacts the count only, or whether we are possibly presenting one recommendation several times to the user (and whether presenting one article multiple times would be a problem or not).

Either way, @Johannnes89 and me agreed that this shouldn't block Add Link from being enabled, as we already have a sufficient number of recommendations.

Related task: T373234: [wmf.21] Special:Homepage - SE counter for articles with structured Add link tasks displays incorrect numbers

Checked in dewiki wmf.24 - the numbers are really close (although they fluctuate, i.e. the differences between them could be bigger) :
Special:Search 29,856
Special:Homepage 29,845
Spezial:NewcomerTasksInfo 29,856

Change #1082043 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] [WIP]: SearchTaskSuggester: avoid duplicates in article count

https://gerrit.wikimedia.org/r/1082043

In T373188#10087643, @Urbanecm_WMF wrote:

I'm curious whether this impacts the count only, or whether we are possibly presenting one recommendation several times to the user (and whether presenting one article multiple times would be a problem or not).

I believe this issue only affects the counter since we run deduplicateSuggestions before creating a task set, so there should be no duplicates within it. And because when the user progresses in the carousel, we also send the excludepageids parameter with the already seen tasks. But theoretically, as things are now, SearchTaskSuggester is not protecting against duplicates between tasksets.

In T373188#10192946, @Etonkovidova wrote:

Checked in dewiki wmf.24 - the numbers are really close (although they fluctuate, i.e. the differences between them could be bigger) :
Special:Search 29,856
Special:Homepage 29,845
Spezial:NewcomerTasksInfo 29,856

Checked again in dewiki wmf.27 and I can consistently reproduce the bug and see an exaggerated counter in the topics selection dialog:

The more I dig into this task I believe it should be considered a feature development rather than a bug. My perception is that this is a defect that has always been present due to the implementation of query per-topic, and the original engineers already noted this solution to be imperfect and filed tasks around how to fix it, entry points T242560: Newcomer tasks: task suggestions fail because of search queries exceeding length limits, T243478: Newcomer tasks: fetch ElasticSearch data for search results. I need to do some more engineer consultation before I can estimate the remaining work to fix this "the right way". That is because changing our search strategy can have side-effects that we want to foresee. Moving to blocked until the implementation plan is clarified.

My observation is that this problem is not restricted to Add a link task but any task in general with enough number of suggestions to be noticeable. The same problem can be seen with Find references and other tasks

In T373188#10087643, @Urbanecm_WMF wrote:

I'm curious whether this impacts the count only, or whether we are possibly presenting one recommendation several times to the user (and whether presenting one article multiple times would be a problem or not).

Suggestions are deduplicated before creating a new task set so that should not happen.

I haven't checked, but I presume this problem is caused by the fact articles and topics have an N:M relationship. If GrowthExperiments is making a separate query for each topic and summing their individual counts together, then we would be counting one article a couple of times, resulting in this confusion.

That seems indeed the case, see SearchStrategy.php#89. The relevant code has had a FIXME comment for a long time pointing to T242560 as the original blocker to not perform a single query for all articles but that task is resolved for some time. All topics together make ~700 characters so we may reach the limit again if the client sens many excluded page ids. But since the limit was doubled since the original implementation it seems worth trying.

After implementing a first version of the multiple topics per query approach, some tests started failing, in particular RemoteSearchTaskSuggesterTest which was asserting the number of requests made is correct and the results returned contain the expected data. Adapting the test, we do now less queries, hence collapsing requests was necessary revealed another problem. Since the search requests are performing one query per topic, our suggester assumes it can add such topic as the "most relevant topic" of the article and assigned the result match score to the topic, eg: art => 0.89268. This information was then added to each task object generated from search results and sent to the client for instrumentation purposes. In particular, the information is meant to be used by the newcomertask schema instruments to feed topic and match_score properties. Probably this data was used in some old GE experiment analysis. However, when looking into the events sent to the relevant schema/stream we find out a bug in the client code that is preventing to add the information into the event payload. As a result, this information has not been present in the ingested events for a long time (haven't determined when but can do if relevant).

For a multi-topic query it is not possible to determine the score for each topic related to a single result, eg: [art => 0.89268, science => 0.19268, ]. In order to do so, changes in the ES index are required, and maybe some changes in CirrusSearch, see T243478. In order to unblock this task, per conversation with @Urbanecm_WMF , we agreed that "decomissioning" topic and match_score properties from the newcomer schema is the simplest way. The topics we could populate per-task are exactly the same as the topic filters would contain, which is possible to get from other streams, eg using the homepagemodule schema. fyi @KStoller-WMF @nettrom_WMF @Iflorez

Filed subtask T380453 to correct this situation and make this task actionable.

Change #1094498 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] TaskSuggester: make search queries multi topic

https://gerrit.wikimedia.org/r/1094498

Change #1098492 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] refactor(SearchStrategy): extract query build methods to functions

https://gerrit.wikimedia.org/r/1098492

Change #1094498 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] TaskSuggester: make search queries multi topic

https://gerrit.wikimedia.org/r/1094498

Change #1098492 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] refactor(SearchStrategy): extract query build methods to functions

https://gerrit.wikimedia.org/r/1098492

Change #1082043 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] SearchStrategy: make one query for ORES topics filters

https://gerrit.wikimedia.org/r/1082043

In T373188#10192946, @Etonkovidova wrote:

Checked in dewiki wmf.24 - the numbers are really close (although they fluctuate, i.e. the differences between them could be bigger) :
Special:Search 29,856
Special:Homepage 29,845
Spezial:NewcomerTasksInfo 29,856

Checked again in dewiki wmf.27 and I can consistently reproduce the bug and see an exaggerated counter in the topics selection dialog:

Thank you, @Sgs for the clarification! The issue appears when the all topic option is actually selected (not when no selection is done). In beta cluster it seems that "Add link" tasks counting issue is fixed.

But, as you mentioned in https://phabricator.wikimedia.org/T373188#10331613, the issue is not restricted to "Add link" task.
For example, on cswiki beta Special:NewcomerTaskinfo page shows the number for section-image-recommendation tasks as 439.
When I do not select any topics on Special:Homepage, I'll see the same number (439 tasks). But if I select all topics, the number fell to be only 211.

I'll check in production and, if some discrepancies would be found for other task types, will file a task.

Checked on dewiki wmf.8 - the task type link-recommendation was selected.

All topic selected	No topic seleceted	Special:Search - hasrecommendation:link	Spezial:NewcomerTasksInfo - link-recommendation
15,929	16,881	16,884	16,884

The difference between the number of available tasks with all topics selected and other two numbers on Special:Search and Special:Newcomer Tasksinfo is relatively small. Is it acceptable to have such discrepancy even as small as it is now?

In T373188#10417790, @Etonkovidova wrote:

The difference between the number of available tasks with all topics selected and other two numbers on Special:Search and Special:Newcomer Tasksinfo is relatively small. Is it acceptable to have such discrepancy even as small as it is now?

The difference is in the right direction now (not all articles need to have a topic attached, so "all topics" might return less articles than "no topics" [as that doesn't engage topic filtering at all]).

hasrecommendation:link -articletopic:africa|central-africa|eastern-africa|northern-africa|southern-africa|western-africa|architecture|visual-arts|asia|central-asia|east-asia|south-asia|southeast-asia|west-asia|biography|biology|business-and-economics|central-america|chemistry|comics-and-anime|internet-culture|software|computing|geographical|earth-and-environment|education|engineering|entertainment|radio|north-asia|eastern-europe|europe|northern-europe|southern-europe|western-europe|fashion|food-and-drink|stem|history|literature|books|mathematics|medicine-and-health|military-and-warfare|music|north-america|oceania|performing-arts|philosophy-and-religion|physics|space|politics-and-government|society|south-america|sports|technology|transportation|films|television|video-games|women indicates 530 articles have a link recommendation but do not have a topic.

Unfortunately, that difference doesn't explain what you see fully. Maybe my query is wrong, and it misses certain articles that match the criteria?

In T373188#10418818, @Urbanecm_WMF wrote:

The difference is in the right direction now (not all articles need to have a topic attached, so "all topics" might return less articles than "no topics" [as that doesn't engage topic filtering at all]).

hasrecommendation:link -articletopic:africa|central-africa|eastern-africa|northern-africa|southern-africa|western-africa|architecture|visual-arts|asia|central-asia|east-asia|south-asia|southeast-asia|west-asia|biography|biology|business-and-economics|central-america|chemistry|comics-and-anime|internet-culture|software|computing|geographical|earth-and-environment|education|engineering|entertainment|radio|north-asia|eastern-europe|europe|northern-europe|southern-europe|western-europe|fashion|food-and-drink|stem|history|literature|books|mathematics|medicine-and-health|military-and-warfare|music|north-america|oceania|performing-arts|philosophy-and-religion|physics|space|politics-and-government|society|south-america|sports|technology|transportation|films|television|video-games|women indicates 530 articles have a link recommendation but do not have a topic.

Unfortunately, that difference doesn't explain what you see fully. Maybe my query is wrong, and it misses certain articles that match the criteria?

Updated my comment above, added the number for "no topics" seelected case.
From a user point of view I'm not sure how to interpret: "not all articles need to have a topic attached, so "all topics" might return less articles than "no topics". What number of tasks I expect to see if I select no topics?