Mirror the current revision of each toolinfo record to an Elasticsearch service. django-elasticsearch-dsl is a likely candidate for implementing this functionality.
Description
Description
Details
Details
Related Changes in Gerrit:
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| search: mirror toolinfo records into Elasticsearch | wikimedia/toolhub | main | +258 -1 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T288685 Establish active/active multi-dc support for Toolhub | |||
| Resolved | • bd808 | T115650 Create an authoritative and well promoted catalog of Wikimedia tools | |||
| Resolved | • bd808 | T271483 Complete and announce initial production deployment of Toolhub | |||
| Resolved | • bd808 | T271376 Discuss production Elasticsearch needs with the Search team | |||
| Resolved | Raymond_Ndibe | T276865 Implement text analysis to support stemming | |||
| Resolved | • bd808 | T195680 Faceted search toolinfo | |||
| Resolved | • bd808 | T271374 Mirror toolinfo records into Elasticsearch |
Event Timeline
Comment Actions
I've looked around a bit more for libraries specifically designed to integrate django apps with full text search backends. The http://haystacksearch.org/ project is nearly ideal in that it attempts to provide a shared service layer that can be fulfilled with multiple backend services. The recency of elasticsearch's SSPL re-licensing (T272238) reminds me that today's perfect fit backend can easily become tomorrow's legacy tech debt that needs immediate attention. Unfortunately, django-haystack seems to have stalled out on the elasticsearch backend side. There is currently no official support for elasticsearch beyond their 5.x release series in django-haystack and upstream issues indicate a lack of maintainership for es integration.
https://django-elasticsearch-dsl.readthedocs.io/en/latest/index.html plus https://django-elasticsearch-dsl-drf.readthedocs.io/en/latest/index.html are the libraries that I'm looking a bit more into now.
Comment Actions
Change 664677 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[wikimedia/toolhub@main] search: mirror toolinfo records into Elasticsearch
Comment Actions
Change 664677 merged by jenkins-bot:
[wikimedia/toolhub@main] search: mirror toolinfo records into Elasticsearch