- Title of session: Semantic Search and RAG on a FOSS stack
- Session description: In this session I will present a prototype software stack to implement Semantic Search and Retrieval Augmented Generation with self-hosted, straight forward and familiar FOSS tools, such as Postgres and Ollama, skipping confusing abstractions such as LangChain and escaping vendor lock in through SaaS products.
- Username for contact: @roti_WMDE
- Session duration (25 or 50 min): 25min
- Session type (presentation, workshop, discussion, etc.): Presentation
- Language of session (English, Arabic, etc.): English
- Prerequisites (some Python, etc.): Interest in deep learning developments, especially LLMs and embeddings.
- Any other details to share?:
- Interested? Add your username below:
Notes:
https://etherpad.wikimedia.org/p/wmh2024-Semantic_Search_and_RAG
notes from session
Semantic Search and RAG on a FOSS stack
Date and time: 4, May, 2024 @ 13:00-13:30
Relevant links
- Phabricator task: https://phabricator.wikimedia.org/T362752
- Session slides: https://rti.github.io/barebone-rag/slides.html
- Source: https://github.com/rti/barebone-rag
Presenter
@[[phab:p/roti_WMDE/|roti_WMDE]]
Participants
- ...
- User:Valerio_Bozzolan
- ....
- Kevin Payravi (User:SuperHamster)
- Jennifer 8. Lee (User:Jenny8lee)
- Markus Glaser (User:Mglaser)
- Mónica (User:Monica_Pinedo_Bajo_(WMDE), /p/darthmon_wmde)
- Damilare
(~17 in total)
Notes
The NVIDIA CUDA cannot be generally recommended to Free Software folks because it works with proprietary drivers. Instead the AMD ROCm stack is MIT licensed and it fits the job.
The LLAMA project is a Free Software mature enough to host a local large language model, and it has a super-easy Python library to just "Hey LLAMA, tell me what this image is about" in few lines of code.
The nightmare is that there are very very few large language models that are reproducible with an Open Source license. See slide 19.
The "Bloom" project started with very good intentions and adopted only contents based on Free Software licenses but then they adopted a proprietary license that tries to impose "good moral choices" using a copyright license that, using copyright, imposes restrictions on the kind of available usages.
See slide 20 for models that are more open than others.