Skip to content

In practice

RAG Explained: Using AI with Your Company Knowledge

A language model doesn’t know your handbooks, price lists or contracts. With retrieval-augmented generation, or RAG, the AI looks up the relevant passages in your documents before every answer. The result: answers based on your knowledge – with citations.

Lurus Editorial Team Published Reviewed 4 min read
Painted scene: a bright desk with a laptop, notes and coffee by an open window.

Key takeaways

  • RAG connects a language model with a search across your own documents.
  • The AI answers based on the passages it finds and can cite them as sources.
  • The model isn’t retrained, and documents can be updated at any time.
  • Answer quality depends directly on the quality and currency of the sources.
  • Access rights must also apply within the AI.
Contents
  1. The problem: the AI doesn’t know your company
  2. The solution: retrieval-augmented generation
  3. How RAG works in four steps
  4. RAG or fine-tuning?
  5. Typical use cases
  6. What matters with the data
  7. Security and access rights
  8. Limits of RAG

The problem: the AI doesn’t know your company

A large language model has learned from publicly available text. It doesn’t know your quality manual, your current price list or your travel expense policy. If you ask about them anyway, you get either no useful answer or – worse – a plausible-sounding but invented one. How such hallucinations arise is explained in AI hallucinations.

The solution: retrieval-augmented generation

Retrieval-augmented generation (RAG) was described in a research paper in 2020: a language model is connected to a searchable collection of documents. The authors name as advantages that knowledge can be updated this way and that answers become more traceable – two weaknesses of pure language models.

How RAG works in four steps

  1. Prepare: Your documents are split into short sections. For each section, a kind of content fingerprint is calculated, known as an embedding.
  2. Search: When someone asks a question, the system finds the sections whose content best matches the question – even if different words are used.
  3. Augment: The sections found are passed to the language model together with the question.
  4. Answer: The model writes an answer based on these sections and can state which document the information comes from.

The model itself isn’t changed. When a new document is added or one is deleted, the effect on answers is immediate.

RAG or fine-tuning?

CriterionRAGFine-tuning
What happens?The model receives relevant passages for the questionThe model is retrained with your own data
Updatingimmediate, by replacing documentsrequires retraining
Citationspossiblegenerally not
Deleting dataremove the documentdifficult, the knowledge sits in the model
Suited forfacts, rules, product knowledgestyle, format, specialized tasks

For most knowledge questions at small and mid-sized companies, RAG is the simpler and more flexible route.

Typical use cases

  • Internal knowledge questions: “How do I claim travel expenses?” – with a reference to the policy
  • Product and service information: technical data, warranty terms, spare parts
  • Sales support: arguments and references from approved documents
  • Onboarding: new employees find processes and contacts faster
  • Document comparison: finding and comparing clauses in contracts or tenders

More examples are in AI use cases for small businesses.

What matters with the data

Answers can only be as good as their sources. These rules have proven themselves:

  • Start small: ten reliable documents are better than a thousand unchecked ones
  • One source of truth: remove outdated versions so that sources don’t contradict each other
  • Assign owners: who keeps which documents up to date?
  • Text, not images: run text recognition on scanned documents first
  • Make dates visible: version dates in documents help put answers in context

Security and access rights

RAG makes knowledge easier to reach – including information not everyone should see. So pay attention to these points:

  • Respect permissions: The AI should only search documents that the person asking can access themselves.
  • Separate sensitive data: HR files or confidential contracts don’t belong in a generally accessible knowledge base.
  • Consider manipulated content: Documents can contain hidden instructions that influence the model. The OWASP list of the most important LLM risks names prompt injection as well as vector and embedding weaknesses as risks of their own.
  • Clarify data protection: If the documents contain personal data, the GDPR applies; see Using AI in compliance with the GDPR.

Limits of RAG

RAG is not a cure-all. The search can find unsuitable sections, important information can be spread across several documents, and the model can summarize what it finds incorrectly. A research survey on hallucinations devotes a separate chapter to the limits of such systems. Citations are therefore more than a convenience: they enable a quick check before an answer is used further. Key terms such as embeddings and vectors are in the AI glossary.

Frequently asked questions

Is RAG the same as training your own model?

No. Training or fine-tuning changes a model’s parameters. With RAG, the model stays unchanged; it simply receives the relevant passages with each request. That means content can be updated or removed at any time.

Which documents are suitable for RAG?

Text-based, current and consistent documents such as handbooks, policies, product documents, FAQs or contract templates. Scanned documents without text recognition or heavily outdated collections lead to poorer answers.

Does RAG prevent hallucinations?

It reduces the risk because the AI answers based on specific sources. It doesn’t rule out errors: the search can find unsuitable passages, and the model can misrepresent content. Citations make checking easier.

Can every employee see every document?

That shouldn’t be the case. A good system respects access rights, so the AI only searches documents the person asking is allowed to open themselves.

Sources

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020) arXiv
  2. A Survey on Hallucination in Large Language Models (Huang et al., ACM TOIS) arXiv
  3. OWASP Top 10 for LLM Applications 2025 OWASP Foundation

Written by the Lurus Editorial Team according to our source standard. Editorial policy

Free initial consultation

Your AI start, planned together.

Show us your tasks. We’ll discuss data paths, rules and the right starting point for your team – no feature tour.

The initial consultation is free of charge. A phone number is not required.