How a Custom Business Chatbot Is Actually Built: Knowledge, Retrieval, Guardrails and Handoff

A custom business chatbot is a language model surrounded by six parts that make it useful and safe:

1. a curated knowledge base;
2. retrieval that finds the right passages for each question;
3. instructions that set scope, tone and rules;
4. tools that let it look up or change records under permission checks;
5. guardrails that check what goes in and comes out;
6. a handoff to a person.

Logging and evaluation run across all six.

The model is the smallest part of the work. Most of the effort goes into preparing knowledge, connecting systems and testing answers — and that is also where the quality comes from.

This article walks through each part in the order a project builds them. Whether you should build at all, or use a platform, is covered in custom chatbot or off-the-shelf platform.

The architecture in one paragraph

A customer writes a message. The system checks it, strips or masks personal data the model does not need, and searches the knowledge base for relevant passages. It sends the model the instructions, the conversation so far, those passages and the list of available tools. The model replies, or asks to use a tool. The system checks permissions, runs the tool and returns the result. The final answer is checked against the rules before the customer sees it. Every step is logged. At any point, the conversation can pass to a person.

1. Knowledge: decide what the bot is allowed to know

A chatbot is only as reliable as the material it answers from. The first job is editorial, not technical.

• Choose sources deliberately. Product information, prices, policies, opening hours and FAQs, each with an owner responsible for keeping it current. Do not feed the bot every document on the shared drive. Contradictory and outdated material produces contradictory and outdated answers.
• Keep public and internal separate. A customer-facing bot should never be able to retrieve internal pricing notes or HR policy. That separation belongs in the data, not just the instructions. An internal AI assistant for your staff covers the internal case.
• Date and version everything. Answers about prices and policies should come from the current version. The bot should be able to say which.
• Write Arabic and English with equal care. If the Arabic knowledge is a thin translation of the English, the Arabic answers will be worse. An Arabic chatbot that understands Gulf customers covers the language side.

2. Retrieval: find the right passage for each question

The technique is called retrieval-augmented generation (RAG), a term from a 2020 research paper. Instead of relying on what the model learned in training, the system looks up passages from your knowledge base for each question. It gives them to the model with an instruction to answer from them.

Three practical points decide whether retrieval works:

• How the knowledge is split. Passages need to be large enough to carry a complete answer and small enough to be specific. A refund policy split mid-sentence produces half-answers.
• How it is searched. Semantic search, which matches meaning, handles paraphrases well. Keyword search handles product codes, names and exact terms. Combining both is usually more robust, particularly in Arabic, where spelling variants and dialect words defeat one method or the other.
• What happens when nothing matches. The bot must be told to say it does not know and offer a person, rather than filling the gap from general knowledge. This is where many invented answers begin.

Keep a reference from each answer back to the passages it used. It makes reviews possible and disputes answerable.

3. Instructions: scope, tone and rules

The instructions — often called the system prompt — tell the model what it is for. Good ones are specific:

• Scope. Which topics it handles, and a polite refusal for everything else.
• Rules. What it may never promise — refunds outside policy, prices not in the knowledge base, legal or medical advice. Also when it must hand over.
• Language behaviour. Reply in the customer’s language; how to handle dialect and mixed messages; when to switch to formal Arabic.
• Disclosure. A first message that says the customer is talking to an AI and can reach a person. In the EU this is a legal duty under the AI Act. Everywhere it is sound practice — see the EU AI Act and Gulf chatbots.

Instructions are guidance to the model, not a security boundary. Customers — or text inside documents the bot reads — can try to override them. This is called prompt injection, and it is first on the OWASP Top 10 for LLM applications. Treat everything the model reads as untrusted, and enforce anything that matters in code.

4. Tools: how the bot acts, safely

Tools turn a chatbot that answers into one that helps. Typical examples are checking an order’s status, rescheduling a booking, or opening a support ticket. Technically, the model asks to call a function with certain parameters, and your system decides whether to run it.

The safety lives in that decision:

• Check permissions on the server. The bot acts as the customer it is talking to, never as an administrator. A customer can see their own orders, not someone else’s, however the request is phrased.
• Confirm before changing anything. “I’ll move your delivery to Thursday 23 October between 4 and 6pm — shall I go ahead?” A clear confirmation protects the customer and you.
• Cap the consequences. Limits on refund amounts, number of changes, and which actions the bot can take at all. OWASP calls the opposite “excessive agency” — a bot with more power than its job needs.
• Log every tool call with inputs, outputs and who it acted for.

Remember the legal weight. Under Oman’s Electronic Transactions Law, a document issued through an automated agent you have set up is treated as issued by you, and contracts can be formed through one. Since September 2026, people can also object to decisions made by automated processing and have a person review them. A bot that can cancel, refuse or approve needs an objection path. When your chatbot gets it wrong goes further.

5. Guardrails: check what goes in and what comes out

Guardrails are checks in code around the model:

• Input checks. Detect and mask personal data the model does not need — national ID numbers, card numbers — before it is sent to an external model. Limit message length, and block known attack patterns.
• Output checks. Validate any price, date or amount against your system before it is shown. Block answers that make promises outside policy, and catch replies in the wrong language.
• Topic limits. Keep the bot on business topics, and decline the rest politely.
• Spend and abuse limits. Cap requests per user, conversation length and daily model spend. OWASP lists “unbounded consumption” as a risk for a reason: a chatbot open to the internet can be made to run up a bill.

No guardrail is perfect, which is why the next part exists.

6. Handoff: a person, at the right moment

A good chatbot knows when to stop. Hand over to a person when:

• the customer asks for one;
• the bot cannot find an answer in the knowledge base;
• the topic is sensitive — a complaint, a medical or legal matter, a vulnerable customer;
• the customer objects to a decision the bot made or applied;
• a tool action fails or needs approval.

Pass the conversation and what the bot already looked up, so the customer does not repeat themselves. Be honest about hours: “A colleague will reply from 8am tomorrow” is better than silence.

On WhatsApp, the platform’s messaging rules shape when a person can reply freely. WhatsApp Business API for customer operations explains them.

7. Logging and evaluation: how you know it works

Log each conversation with the passages retrieved, the tools called and the final answer. That lets you answer three questions: what did the bot say, why, and what did it do. It is also what you will need if a customer disputes an answer.

Logs are personal data. Give them a retention period tied to a purpose and erase them afterwards — Oman’s law has required erasure when the purpose ends since September 2026. What your chatbot does with customer data covers the rules.

Then evaluate on purpose:

• Build a test set from real questions, in Arabic and English, with the correct answers agreed by the business.
• Run it before every change — new knowledge, new instructions, a new model — to catch regressions.
• Review a sample of live conversations every week, marking wrong, incomplete and unnecessarily escalated answers.
• Track a few measures: how many conversations resolve without a person, how many are handed over and why, and how many wrong answers the reviews find.

A sensible first version

Start narrow. Answer questions from a well-maintained knowledge base, add one or two actions that matter — order status, booking changes — and a reliable handoff. Run it, read the logs, and expand from what customers actually ask. A first version that does three things well earns trust that a version promising everything never gets.

---

Frequently asked questions

What is RAG in a chatbot? Retrieval-augmented generation: for each question, the system searches your knowledge base and gives the relevant passages to the model, which answers from them. It keeps answers tied to your content rather than to what the model learned in training.

Can a chatbot be connected to our ERP or booking system? Yes, through tools — functions the model can ask to call, which your system executes after checking permissions. Design them to act only on behalf of the customer in the conversation, to confirm before changes, and to log every call.

How do we stop the chatbot inventing answers? Instruct it to answer only from retrieved passages, and to say it does not know otherwise. Check prices and amounts against your system before display, and keep the knowledge base current. Review real conversations regularly to catch what slips through.

What is prompt injection? An attempt — by a user or through text in a document — to override the bot’s instructions. It is the top risk in OWASP’s list for LLM applications. Do not rely on instructions for security; enforce permissions and limits in code.

Does the chatbot need to hand over to a human? Yes. Customers need a route to a person for complaints, sensitive matters and anything the bot cannot answer. In Oman, people may also object to automated decisions, and a person must review the decision.

How long should we keep chatbot conversation logs? For a defined purpose, such as dispute handling and quality review, and no longer. Oman’s data protection law requires erasure once the purpose ends, unless a dispute or legal obligation requires keeping them.

---

This article states the position under the laws in force on 21 October 2026, the date of publication, and is not legal advice.

Sources: OWASP — Top 10 for LLM Applications · Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020) · Royal Decree 68/2026 amending the Personal Data Protection Law · Royal Decree 39/2025 — Electronic Transactions Law (Arabic text) · AI Act Article 50 — transparency obligations

Building a chatbot that answers from your knowledge and acts in your systems? ZAWAT builds AI and automation and the integrations behind it, with guardrails, handoff and logging designed in. Book a call.