For answers that trace back to a real document
RAG Implementation
A model answering purely from memory drifts, forgets updates and occasionally states a wrong fact with total confidence. Retrieval-augmented generation removes that risk by fetching the actual passage before the model responds, so every claim traces to a real document. This practice builds the retrieval pipeline itself, because chunking and index quality decide whether an answer is trustworthy.
How a grounded answer actually gets produced
Chunking, retrieval ranking and the groundedness check, walked through end to end.
80 secSample frame · video pending01Before any pipeline gets built
Retrieval solves a specific problem. Confirm it is yours.
Retrieval fixes a grounding problem, not a reasoning problem, and mistaking one for the other wastes a build. Four questions settle which risk you are actually carrying.
Does the answer need to cite something real?
If users need a source they can check, retrieval is the right mechanism. If they need judgment applied to a case, that is a different problem entirely.
How often does the source content actually change?
Fast-moving content favours retrieval, because reindexing is cheaper than retraining a model every time a document is updated.
Can retrieval latency fit your response budget?
An extra lookup step adds time before generation starts. Interactive products need that cost measured before launch, not discovered after.
Does content carry access restrictions per user?
Retrieval must respect who is allowed to see what, at query time, or it becomes a new way to leak restricted content.
| Prompt stuffing | Retrieval (RAG) | Fine-tuning | |
|---|---|---|---|
| Content freshness | Manual, per prompt | Reindex and it updates | Requires retraining |
| Cost to update | Cheap, small scale only | Moderate, scales with corpus | Highest, needs a run |
| Answer traceability | Weak | Strong, cites the passage | Weak, baked into weights |
| Right for | Tiny, static content | Large or changing corpora | Style, tone, reasoning shape |
02Fixed before the pipeline is built
What stays yours while the pipeline gets built
Every retrieval engagement draws the same line: the content you bring, what this practice builds around it, what comes back to you, and what never moves outside your control.
- Your side
You bring
- The assistant and its current prompt design
- The document set answers should be grounded in
- Examples of answers that went wrong
- Our side
We build
- A chunking and indexing strategy for that content
- A retrieval step wired ahead of generation
- A groundedness check on every produced answer
- Your side
You receive
- An assistant that cites the passage behind each claim
- An evaluation harness scoring groundedness over time
- A dashboard showing where retrieval still misses
Never leaves you
- Ownership of the source documents and their access rules
- The product experience around the assistant
- Approval of what counts as an acceptable answer
Where the line sitsWe carry the retrieval pipeline. The content stays yours, and so does the judgment call on what a correct answer looks like.
03From raw content to a cited answer
Four stages, each closing on evidence
Retrieval systems fail quietly when chunking or ranking choices get made by instinct. This system makes each choice explicit and leaves an artefact behind it.
Map the corpus and its boundaries
Document types, update frequency and access tiers are catalogued before any chunking decision gets made.
A content map with access rules attachedBuild chunking and retrieval
Chunk size, embedding choice and ranking are tuned against representative queries, not a default configuration lifted from a tutorial.
A retrieval step tuned against real queriesTest groundedness adversarially
The system is pushed with queries designed to expose ungrounded answers before real users ever see one.
An evaluation set drawn from real failuresHand over monitoring and refresh
Reindexing triggers, groundedness dashboards and the escalation path move to your team at handover.
A pipeline your team can operate
04What actually drives the number
Four drivers, not a flat rate card
Retrieval work is priced against the assessment, because these four factors change the effort by multiples, not by a small margin.
Corpus size and format variety
A single document type indexes cheaply. Mixed formats, scanned files and structured data each add distinct extraction work.
Update frequency of source content
Static archives index once. Live content needs a refresh pipeline built and monitored as part of the system.
Access control complexity
Per-user or per-tenant restrictions require retrieval to filter at query time, which most default setups never handle.
Evaluation rigor required
A support assistant tolerates more variance than a system quoting policy or contract terms. The evaluation bar scales the effort.
05Evidence and proof
Answers that hold up when a customer checks them.
Retrieval proof is groundedness and deflection, not fluency scores. Every record here waits for written client approval before production.
Explore case studiesSupport answers started citing the actual policy line.
Koru Health's support assistant answered from memory and occasionally misstated coverage details. A retrieval layer over the policy library grounded every answer in a citable passage.
- Answers with a cited source
- Majority of sessions
- Escalations for wrong answers
- Sharply reduced
One index replaced four disconnected search tools.
Aster North ran separate search tools per product line, each returning inconsistent results. A unified retrieval index gave every product the same grounded answer engine.
- Search tools consolidated
- Four into one
- Cross-product answer consistency
- Markedly improved
We stopped arguing about whether an answer was right. Now we can just open the cited passage and check.
Sample · pending client approval
06Read this first
When retrieval is the wrong spend
Selling a retrieval pipeline to a team that needs reasoning over a handful of documents costs more than it earns in trust. The exits are marked before the pitch. When the build is justified, it runs on vector and hybrid search infrastructure, with your existing model provider underneath.
The content set is small and static
A few dozen stable documents often fit inside a prompt directly. A retrieval pipeline solves a scale problem you may not have.
- Large language model development
The task needs reasoning, not lookup
Consistent tone, style or domain reasoning is a fine-tuning question, not a retrieval one.
- Team extension
Your team should own this build
If your engineers already understand embeddings and just need capacity, team extension gets you there without an added delivery layer.
| Shape | Best when | You direct | We carry |
|---|---|---|---|
| Scoped build | The corpus and access model are settled | Priorities and acceptance | Retrieval pipeline delivery |
| Delivery pod | The corpus will keep expanding | Product decisions each cycle | Engineering, evaluation, releases |
| Team extension | Your leads run delivery well | Architecture and standards | Named engineers and continuity |
07Straight answers
Asked before a retrieval build starts.
01How is RAG different from fine-tuning?
Retrieval fetches the relevant passage at query time and leaves the model's weights untouched, while fine-tuning changes the weights themselves through additional training. Retrieval suits content that changes often; fine-tuning suits a fixed style or reasoning pattern that should not need a lookup.
02Can RAG and fine-tuning work together?
Yes, and the combination is common once the retrieval layer is proven. A fine-tuned model can still call retrieval for facts while relying on its trained weights for tone and domain reasoning.
03What does chunking mean in a retrieval pipeline?
Chunking splits documents into passages small enough to retrieve precisely but large enough to keep their meaning intact. Poor chunking is the most common cause of a retrieval system returning irrelevant context.
04How do you measure groundedness?
Against a curated set of real questions with known correct passages, scoring whether each answer cites and reflects the retrieved content accurately. The set grows from logged failures, not a fixed benchmark alone.
05Can retrieval respect per-user access restrictions?
Yes, when the retrieval layer is built to filter by permission at query time rather than trusting the content store to have already restricted it. This is engineered explicitly, not assumed.
06What does a RAG implementation cost?
Scoped engagements are priced against corpus size, format variety and access complexity from the assessment. Ongoing delivery runs as a monthly team commitment sized to your content roadmap.
07Do you work with our existing model provider?
Yes, retrieval is built as a layer in front of the model you already use, so switching providers later does not require rebuilding the pipeline underneath it.