Page MenuHomePhabricator

Figure out mailman3 search index config
Closed, ResolvedPublic

Description

Mailman3 supports fulltext search, a long-requested for mailing lists. There are multiple backends which are supported. For now I'd like to avoid elasticsearch since it adds in an extra dependency on another service, so realistically we're looking at xapian or whoosh (pure Python).

xapian is currently enabled and rough estimates look like it'll be a 212 GB folder. (From T278609#6982087) but that estimate might be inflated because of attachments. IIRC whoosh is a smaller index but slower indexing.

Of course we should also look at quality of search results.

Event Timeline

fgiunchedi triaged this task as Medium priority.Apr 13 2021, 1:00 PM
Legoktm claimed this task.

xapian seems to be working fine for now.

root@lists1001:/var/lib/mailman3/web# du -hs fulltext_xapian_index/
50G	fulltext_xapian_index/

Of course we should also look at quality of search results.

I haven't heard any complaints yet, and in my searches it's been able to find what I've been looking for.