RAG vs fine-tuning: when to use which in your chatbot

Every two weeks someone calls us asking how much it would cost to "train an AI on our company's data." Almost always the right answer is: you probably don't need to train it — you need it to consult your data. And the cost difference between the two approaches is five digits.

In the world of AI chatbots and assistants, there are two main paths: RAG and fine-tuning. Many businesses confuse them, and agencies operating in murkier waters charge for expensive fine-tuning when RAG was all the client ever needed. Let's set the record straight — not on implementation details, but on what each approach is actually for.

What each approach does, in plain English

Fine-tuning means teaching the AI new things deeply: you take a base model and train it on your data until it integrates that knowledge into its reasoning. It's expensive, slow, and done wrong it damages the model. Done right it's powerful — but only makes sense in very specific cases.

RAG — retrieval-augmented generation — is a different thing entirely. You leave the model as-is but give it real-time access to your data every time it's asked something. It searches your documents, pulls in what's relevant, and responds. It's like giving a library card to a brilliant senior consultant rather than making them memorize every book in the building.

The analogy we use with clients: fine-tuning is hiring a new employee and putting them through a year of onboarding from scratch. RAG is giving a senior employee the keys to the company archive. For most small- and medium-business situations, the second is faster, cheaper, and better.

Three questions before choosing

When a client brings us a chatbot project, we ask three questions. The answers decide which path to take.

Does the data change frequently? If your product catalog, prices, stock, or content updates weekly, RAG is the only sensible option. Fine-tuning weekly is economically unworkable. A restaurant changes its menu, an e-commerce changes stock, a consultancy adds case studies every month — all of that points to RAG.

Does the chatbot need to adopt a very specific tone or style? Here fine-tuning starts to make sense. If you need the bot to respond exactly the way your brand speaks — your vocabulary, your turns of phrase, your register — and that's critical, fine-tuning can help. But watch out: in many cases the same effect is achievable with a well-crafted system prompt and RAG, at about 5% of the cost.

Does the query volume justify the fixed cost? Fine-tuning makes economic sense when you're handling hundreds of thousands of queries per month. Below that, almost never. An SMB with three hundred queries a month doing fine-tuning is like buying a truck to pick up a loaf of bread.

The most expensive mistake we see

Small businesses paying twenty thousand euros to a consultancy for a "custom AI training," when what they needed was a RAG pipeline connected to their document base — something that costs a fraction of that and produces equivalent or better results.

The mistake usually comes from marketing: "fine-tuning" sounds more sophisticated. RAG sounds like a dull acronym. But sophistication isn't a KPI — results are. And the quality-to-cost ratio of RAG, for the vast majority of typical business use cases (retailers, restaurants, hotels, professional practices, service websites), is unbeatable.

Why we almost always recommend RAG

When we build chatbots for SMBs and local businesses, RAG wins by a mile. Specific reasons:

Automatic updates: if your menu, catalog, or hours change, the bot knows the next day — without retraining anything.

Traceability: every bot response can cite its exact source within your data. This is critical in regulated sectors and, in general, builds trust with the person asking.

Predictable cost: you pay per query, not a large fixed training fee. That lets you start small and scale.

Speed to launch: a solid RAG setup can be live in days, not months.

When fine-tuning actually makes sense

To be fair: there are cases. Customer service bots for large enterprises with a strongly defined brand voice. Specialized assistants in technical sectors where vocabulary is critical (legal, medical, financial). Products where the model is sold as the company's proprietary tech and latency needs to be minimal. All of that can justify fine-tuning, or a hybrid approach.

For an SMB or local business, it almost never is the right path. And if someone recommends it without first ruling out RAG, they're probably selling the more expensive product, not the most appropriate solution.

At A-Digital we've been building RAG-based chatbots for clients in Menorca and across Spain for years. From restaurant bots on the island to support assistants for e-commerces selling across Europe. The conclusion, after many projects, is that 95% of what an SMB needs from a chatbot is solved well by RAG. And if the case genuinely requires fine-tuning, we'll tell you — and explain why.