Retrieval-Augmented Generation (RAG) is the currently dominant architectural pattern for providing a large language model with current, company-internal knowledge without retraining the model itself. A RAG pipeline fetches relevant text snippets from a vector database with each request and passes them to the LLM along with the user’s query as context. The model then generates the answer based on this context – rather than based on its static training knowledge.
Context
In the ERP environment, RAG is particularly valuable because company knowledge comes from heterogeneous sources: SAP B1 manuals, CoSIT specifications for e-invoicing, DATEV documentation, internal wikis, customer requirement specifications, Coresuite/VFS documentation, blog posts, support cases. A typical pipeline: (1) Source documents are converted into Portions disassembled, (2) mapped into a 768-dimensional vector space using an embedding model (e.g. gemini-embedding-001), (3) stored in a vector database (Supabase with pgvector, Pinecone, Weaviate). At runtime, the user's question is also converted into a vector, and the most similar chunks are retrieved. Cosine similarity retrieved and incorporated as context into the LLM prompt. Advanced RAG variants combine vector search with full-text search (hybrid search), use rerankers, or employ a multi-step approach.Agentic RAGIn products like B1-Helpster, RAG is combined with direct service layer connectivity, enabling the agent to answer conceptual questions as well as query live data.
Demarcation
RAG is not fine-tuning: it doesn't change the model but feeds it contextually. Compared to pure prompt engineering, RAG is dynamic and scales with the size of the knowledge base. Compared to classic search, it provides not just links but a synthesised answer – but carries the risk of the model misinterpreting the sources. Clean citations, traceable source references, and a clear „I don't know" when retrieval is empty are therefore mandatory components of any ERP RAG implementation.
ReAct and RLEF: How Artificial Intelligence Learns from Real Mistakes — and What Automated Bank Reconciliation in SAP Business One Has to Do with It
SAP B1 10.0 FP2608: Service Layer AI as a transactional layer
Process Reward Models: Why a correct result does not yet prove a correct method
Test-Time Compute Scaling: Why a Smaller AI Model Can End Up Winning — and What Supplier Comparison in SAP Business One Has to Do With It
RLHF and reward models: AI hype or what the approval process in SAP Business One has to do with it
