Page MenuHomePhabricator

Active user accounts don't show up in fulltext search when setting "Document Status" to "open" (and email verification is required)
Open, Stalled, LowPublicBUG REPORT

Description

Upstream ticket: https://we.phorge.it/T16136

Steps to replicate the issue (include links if applicable):

  • Go to "Advanced search".
  • In "Query" enter "Anna_Jegeus_WMSE".
  • For "Document Status" check "Open".
  • In "Document Types" enter "User"
  • Click "Search".

What happens?:
There are no results.

What should have happened instead?:
User @Anna_Jegeus_WMSE shows up in the search results.

I may have misunderstood something, but I read "open" as active for user accounts. This seems to be the case for most accounts when I search "WMSE", but there are a few more that don't show up, e.g. @Andreas_Kettelhoit-WMSE and @Sandra_Fauconnier_WMSE. Is there another definition of what's "open" and "closed" here?

Other information (browser name/version, screenshots, etc.):
Firefox 125.0.3, Ubuntu 22.04.4 LTS.

Event Timeline

Aklapper edited projects, added Phabricator (Search); removed Phabricator.

It works for me when not setting the document type. (And I cannot reproduce in upstream.)

Aklapper renamed this task from Active user accounts don't show up in advanced search when "Document Status" is "open" to Active user accounts don't show up in advanced search when setting "Document Type" to "User" (and "Document Status" to "open").May 16 2024, 12:10 PM

When selecting Document Status = open, the resulting SQL query has an additional condition user_user_fdocument.isClosed = 0.
That is false for e.g. Anna in the DB having isClosed = 1, while other users which are correctly found have isClosed = 0:

mysql:phstats@m3-slave.eqiad.wmnet [phabricator_user]> SELECT u.userName, f.* FROM user_user_fdocument f INNER JOIN user u ON u.phid = f.objectPHID WHERE f.objectPHID = "PHID-USER-gjeg26or67pbqd6rk2aw" OR f.objectPHID = "PHID-USER-j2ysgbht5lbabsrdyvwm";
+------------------+-------+--------------------------------+----------+------------+-----------+--------------+---------------+
| userName         | id    | objectPHID                     | isClosed | authorPHID | ownerPHID | epochCreated | epochModified |
+------------------+-------+--------------------------------+----------+------------+-----------+--------------+---------------+
| Eric_Luth_WMSE   | 17393 | PHID-USER-gjeg26or67pbqd6rk2aw |        0 | NULL       | NULL      |   1472458662 |    1540903683 |
| Anna_Jegeus_WMSE | 41452 | PHID-USER-j2ysgbht5lbabsrdyvwm |        1 | NULL       | NULL      |   1707296891 |    1707296891 |
+------------------+-------+--------------------------------+----------+------------+-----------+--------------+---------------+

ft_doc.isClosed is read as a parameter in PhabricatorCursorPagedPolicyAwareQuery.

It seems isClosed is written in https://phabricator.wikimedia.org/source/phabricator/browse/wmf%252Fstable/src/applications/search/engineextension/PhabricatorFerretFulltextEngineExtension.php$26-48 which sets PhabricatorSearchRelationship::RELATIONSHIP_CLOSED for some reason.

That reason comes from $type in PhabricatorSearchAbstractDocument::addRelationship($type, $related_phid, $rtype, $time).

I assume that's called in PhabricatorUserFulltextEngine::buildAbstractDocument which checks for

$user->isUserActivated()
  ? PhabricatorSearchRelationship::RELATIONSHIP_OPEN
  : PhabricatorSearchRelationship::RELATIONSHIP_CLOSED,

PhabricatorUser:isUserActivated() checks if an account is "enabled, verified (if required), approved (if required)" and returns false if not.
We require validation (means: email) per https://phabricator.wikimedia.org/config/edit/auth.require-email-verification/ : "email addresses must be verified (by clicking a link in an email) before a user can login".

So I wonder if this account got indexed (not sure how that is triggered) before it got "verified".
And whether there is any code to ever reindex when verification happened in the meantime.

That entry in ftdoc was created at 1707296891. It does not look like Phab logs when an account got verified though.

Aklapper renamed this task from Active user accounts don't show up in advanced search when setting "Document Type" to "User" (and "Document Status" to "open") to Active user accounts don't show up in fulltext search when setting "Document Status" to "open".Jun 29 2025, 2:03 PM
Aklapper renamed this task from Active user accounts don't show up in fulltext search when setting "Document Status" to "open" to Active user accounts don't show up in fulltext search when setting "Document Status" to "open" (and email verification is required).
Aklapper updated the task description. (Show Details)
Aklapper moved this task from Backlog to Reported Upstream on the Upstream board.
Aklapper moved this task from Backlog to Upstreamed on the Phabricator (Upstream) board.
Aklapper updated the task description. (Show Details)
Aklapper removed a subscriber: Sandra_Fauconnier_WMSE.
Aklapper changed the task status from Open to Stalled.Jul 6 2025, 9:40 PM
Aklapper claimed this task.
Aklapper moved this task from Reported Upstream to Patch proposed upstream on the Upstream board.

This should first get fixed upstream, then we need to pull / deploy a more recent upstream, then I need to reindex all users in our installation

I ran
SELECT u.userName fdoc FROM phabricator_user.user_user_fdocument f INNER JOIN phabricator_user.user u ON u.phid = f.objectPHID WHERE f.isClosed = 1 AND u.isDisabled = 0 AND u.isEmailVerified = 1 AND u.isApproved = 1 AND u.isMailingList = 0;
and then I reindexed the resulting users via
sudo ./bin/search index --force --background @Alice @Bob @Charlie @etc

Obviously won't fix the next user to register an account (see my previous comment) but global search results for open users should be rather complete for now at least.

Note to myself: I finally got to the root problem and proposed https://we.phorge.it/D26553 in upstream.
Once pulled and deployed in our instance, still need to fix the existing DB entries manually.