Retrieval-Augmented Generation (RAG) is the currently dominant architectural pattern for providing a large language model with current, company-internal knowledge without retraining the model itself. A RAG pipeline fetches relevant text snippets from a vector database with each request and passes them to the LLM along with the user’s query as context. The model then generates the answer based on this context – rather than based on its static training knowledge.
Context
In the ERP environment, RAG is particularly valuable because company knowledge comes from heterogeneous sources: SAP B1 manuals, CoSIT specifications for e-invoicing, DATEV documentation, internal wikis, customer requirement specifications, Coresuite/VFS documentation, blog posts, support cases. A typical pipeline: (1) Source documents are converted into Portions disassembled, (2) mapped into a 768-dimensional vector space using an embedding model (e.g. gemini-embedding-001), (3) stored in a vector database (Supabase with pgvector, Pinecone, Weaviate). At runtime, the user's question is also converted into a vector, and the most similar chunks are retrieved. Cosine similarity retrieved and incorporated as context into the LLM prompt. Advanced RAG variants combine vector search with full-text search (hybrid search), use rerankers, or employ a multi-step approach.Agentic RAGIn products like B1-Helpster, RAG is combined with direct service layer connectivity, enabling the agent to answer conceptual questions as well as query live data.
Demarcation
RAG is not fine-tuning: it doesn't change the model but feeds it contextually. Compared to pure prompt engineering, RAG is dynamic and scales with the size of the knowledge base. Compared to classic search, it provides not just links but a synthesised answer – but carries the risk of the model misinterpreting the sources. Clean citations, traceable source references, and a clear „I don't know" when retrieval is empty are therefore mandatory components of any ERP RAG implementation.
AI in everyday work: Here's how to find the right start
AI assistant meets SAP Business One: Two paths out of the Versino AI shop
Sprint and Swirl: How an AI agent divides work and learns from trials
LATS: Considering several paths simultaneously with AI — and the availability check in SAP B1
ReAct and RLEF: How Artificial Intelligence Learns from Real Mistakes — and What Automated Bank Reconciliation in SAP Business One Has to Do with It
