[Workshop] Building Privacy-First Vector Search Pipelines With Local LLMs

Abstract

Most AI-powered search today depends on cloud services, which creates two major challenges: data exposure and vendor lock-in. Sensitive information often leaves local environments, and organizations lose control over how their data is stored, processed, and queried. This workshop presents a practical alternative: building privacy-first vector search pipelines using containerized infrastructure and locally deployed Large Language Models. We explore multiple aspects of designing Retrieval-Augmented Generation (RAG) systems that are fully vector-database agnostic, allowing teams to choose or switch embedding stores without being tied to a single vendor. By running local LLMs such as Qwen3 and GPT-OSS with tools like Ollama and OpenCode, teams can support large-context queries (100K+ tokens) while keeping all data entirely within their own environments. The session covers local LLM setup, embedding workflows, indexing strategies, query orchestration, and secure inference patterns. We also demonstrate how to ground responses using selective web retrieval for up-to-date context without compromising privacy, data sovereignty, or regulatory requirements. Scheduled for Wednesday, 23 September 2026 at 5:10 PM in Workshops in-person 3, under the Data Science / AI track. Attendees leave with practical insights from real-world implementations, enabling them to design secure, offline-capable, privacy-first GenAI search platforms suitable for enterprise, regulated, and edge environments.