See: T157783: Searching open tasks found closed tasks

Hmm. So in theory mentioning T363858: MenuButton: Introduce a WIP MenuButton component to Codex fixes the index? (per T157783#5744922)

In T397392#10931229, @kostajh wrote:

Hmm. So in theory mentioning T363858: MenuButton: Introduce a WIP MenuButton component to Codex fixes the index? (per T157783#5744922)

If so, that didn't seem to work: T363858 still shows up in https://phabricator.wikimedia.org/search/query/HVbGB2UrQnjc/#R

[OT] @kostajh: Could you explain how/why you end up on the Global/Primary Search page instead of Maniphest's Advanced Search?
Are you coming from Logstash, or somewhere else?

In T397392#10936643, @Aklapper wrote:

[OT] @kostajh: Could you explain how/why you end up on the Global/Primary Search page instead of Maniphest's Advanced Search?
Are you coming from Logstash, or somewhere else?

I'm not sure that I understand -- isn't https://phabricator.wikimedia.org/search/query/HVbGB2UrQnjc/#R the Advanced Search? I searched for "MenuButton" and then filtered to include only "Open" status and "Tasks".

In T397392#10936696, @kostajh wrote:

I'm not sure that I understand -- isn't https://phabricator.wikimedia.org/search/query/HVbGB2UrQnjc/#R the Advanced Search?

Uhm. No it is not (and I wonder how to make that more obvious to more users):

Annoyingly (from a UI/user understandability perspective) they both seem to be called "Advanced Search" - one being the all-application "Advanced Search", and the other being the task-specific "Advanced Search".

If I was going to suggest a wild idea, maybe it's worth investigating the possibility of forwarding the top-right search bar to the task-specific Advanced Search by default (with "Group By" set to None & "Order By" set to Relevance), on the massive assumption that most people using it will be looking for Phabricator tasks only (and also that the task-specific search doesn't provide too much worse of a user-experience)? But that would definitely require consultation beforehand in case that assumption proves to be wrong, and might also need a fair amount of code adjustments (I haven't checked).

@A_smart_kitten: Right, see T397558. (Maybe we should stop saying "Global" and "Advanced Search" but instead "Primary" and "App-specific Search".)

@A_smart_kitten: Oh well, that would be T288905: Allow Phabricator Current Application context of Advanced Search & always display a Global Advanced Search.

This is a similar underlying problem as T365128: Active user accounts don't show up in fulltext search when setting "Document Status" to "open" (and email verification is required):
ManiphestTaskFulltextEngine checks for PhabricatorSearchRelationship::RELATIONSHIP_OPEN or _CLOSED.
But open tasks and some resolved tasks have the same isClosed = 0 status in the search index, for the latter that's incorrect.
Picked two example tasks, one closed one open, showing up in the global fulltext search results:

mysql:phstats@m3-slave.eqiad.wmnet [phabricator_maniphest]> SELECT t.id, f.isClosed FROM maniphest_task_fdocument f INNER JOIN maniphest_task t ON t.phid = f.objectPHID WHERE (t.id = 379299 OR t.id = 311306);
+--------+----------+
| id     | isClosed |
+--------+----------+
| 311306 |        0 |
| 379299 |        0 |
+--------+----------+
2 rows in set (0.001 sec)

But T379299 (PHID-TASK-wfsl3vlfkb3evivwk2k3) is closed. But that very global fulltext search DB table value doesn't think so.

I remain clueless how exactly reindexing is triggered and why it sometimes fails.
buildAbstractDocument() is called in PhabricatorFulltextEngine::buildFulltextIndexes(),
that's called in PhabricatorFulltextIndexEngineExtension::indexObject(),
that's called in PhabricatorIndexEngine::indexObject() (same name, different signature),
that's called in PhabricatorSearchWorker::doWork(),
that's called in PhabricatorWorker::executeTask(),
that's called in PhabricatorWorker::scheduleTask().

Notes to myself:

Getting affected tasks (resolved in the task DB but not in the search DB):
SELECT t.id, t.status, f.isClosed FROM phabricator_maniphest.maniphest_task_fdocument f INNER JOIN phabricator_maniphest.maniphest_task t ON t.phid = f.objectPHID WHERE (t.status != "open" AND t.status != "stalled" AND t.status != "progress") AND f.isClosed = 0;

Getting affected tasks (not resolved in the task DB but resolved in the search DB):
SELECT t.id, t.status, f.isClosed FROM phabricator_maniphest.maniphest_task_fdocument f INNER JOIN phabricator_maniphest.maniphest_task t ON t.phid = f.objectPHID WHERE (t.status = "open" OR t.status = "stalled" OR t.status = "progress") AND f.isClosed = 1;
This latter query has zero results.