When Your Chatbot Gets It Wrong: Liability, Invented Answers and the Guardrails That Contain Them
If your chatbot tells a customer something wrong — a price, a refund rule, a delivery promise — assume your business is responsible for it. The best-known case is from Canada. In February 2024 a British Columbia tribunal held Air Canada liable for its website chatbot’s wrong answer about bereavement fares. It rejected the airline’s argument that the chatbot was responsible for its own words: “While a chatbot has an interactive component, it is still just a part of Air Canada’s website.”
Omani law points the same way. The Electronic Transactions Law treats a document issued through an automated agent you have set up as issued by you, and allows contracts to be concluded through one. Since September 2026, people can also object to decisions made by automated processing and have a person review them.
The practical answer is not a disclaimer. It is a chatbot designed so that it cannot easily invent commitments: it answers from your sources, checks figures against your systems, confirms before acting, and hands over when unsure.
The short answer
- You answer for your chatbot. Courts and tribunals have treated it as part of the business, not a separate speaker.
- In Oman, its messages are yours. Article 12 of the Electronic Transactions Law attributes documents issued through your automated agent to you.
- It can bind you. Article 19 of the same law permits contracts concluded through automated agents.
- A link to the right policy is not a defence if the bot’s own words contradict it — that was the situation in the Air Canada case.
- Design is the control: grounded answers, validated figures, structured policies, confirmation before commitments, and a human route.
The case everyone cites
The facts of Moffatt v. Air Canada are simple. A customer asked the airline’s website chatbot about bereavement fares after a death in the family. The chatbot told him he could buy a full-fare ticket and apply for the bereavement discount afterwards. The airline’s actual policy, on a page the chatbot linked to, did not allow retroactive claims. When he applied, the airline refused.
The airline argued, among other things, that it could not be held liable for information provided by its chatbot. The British Columbia Civil Resolution Tribunal rejected that. It found the airline owed the customer a duty of care and had not taken reasonable care to ensure its chatbot was accurate, which amounted to negligent misrepresentation. It awarded CAD 650.88 in damages — the difference between what he paid and the bereavement fare — plus interest and fees, about CAD 812 in total.
The amount was small. The reasoning is what matters. The tribunal treated the chatbot as part of the website, and the airline as responsible for what its website said, whether it appeared on a static page or in a chat. It also did not accept that the customer should have checked the chatbot’s answer against another page of the same website.
This is a decision of a Canadian small-claims tribunal and binds no one in Oman. But it is the clearest public statement of how a decision-maker is likely to see a business chatbot, and it matches the direction of Omani law.
Why Omani law points the same way
Attribution. Article 12 of the Electronic Transactions Law issued by Royal Decree 39/2025 treats an electronic document as issued by the originator when it comes from an automated electronic agent the originator has set up to work automatically. A chatbot reply is an electronic document. Under that article, it is your reply.
Contracts. Article 19 allows contracts between an automated agent and a person who knows, or is presumed to know, that they are dealing with one. A chatbot that takes orders, confirms bookings or agrees to refunds can create obligations in both directions.
Automated decisions. Since 7 September 2026, Article 14 of the Personal Data Protection Law has given people the right to object to decisions resulting from automated processing. The controller must then bring a human into the review. A chatbot that refuses a refund or cancels a booking makes exactly that kind of decision. Breaching Article 14 carries a fine of OMR 5,000 to 10,000. What Royal Decree 68/2026 changed covers the provision.
None of this needs a specific AI law. It follows from rules about electronic documents, contracts and personal data that already apply to your business.
How chatbots get it wrong
Knowing the failure modes tells you where to put the controls:
- No answer found, so it improvises. Retrieval returns nothing relevant and the model fills the gap from general knowledge. This is how a plausible but invented policy appears.
- Outdated knowledge. The price list changed and the knowledge base did not.
- A misread question. Dialect, mixed language or an ambiguous phrase leads to the right answer to the wrong question. An Arabic chatbot that understands Gulf customers covers this.
- Over-generalising a rule. “Returns within 14 days” becomes “you can return anything within 14 days”, including the exceptions.
- Arithmetic and figures. Language models are unreliable calculators. Totals, discounts and dates computed by the model can be wrong.
- Manipulation. A user persuades the bot to ignore its instructions — prompt injection — and it agrees to something it should not.
- Tool errors. The bot acts on the wrong record, or acts without the customer’s confirmation.
The guardrails that contain it
Answer from sources, or not at all. Instruct the bot to answer only from retrieved passages, keep a reference to the passage each answer used, and say “I don’t know — let me connect you with a colleague” when nothing relevant is found.
Check every figure against your system. Prices, totals, dates and stock should come from your systems through tools, not from the model’s text. Validate any figure in the final answer before the customer sees it.
Turn policies into rules, not just prose. Refund eligibility, delivery windows and warranty periods are better encoded as rules the system evaluates, with the model explaining the result. A policy the model must interpret from a paragraph is one it can misinterpret.
Confirm before committing. Before any action that creates an obligation — a booking, a cancellation, a refund — restate it and ask for confirmation. Record the confirmation.
Limit what the bot can promise. Make it a rule that the bot only states commitments that exist in policy. Anything else — exceptions, goodwill gestures, compensation — goes to a person.
Hand over at the right moments. Complaints, disputes, sensitive matters and any objection to a decision should reach a person, with the conversation attached.
Treat everything the model reads as untrusted. Customer messages and retrieved documents can contain instructions. Permissions and limits must be enforced in code, never only in the prompt. How a custom business chatbot is actually built covers each of these components.
The risks on OWASP’s list
The OWASP Top 10 for LLM Applications (2025) names the risks security teams now test for. Six map directly onto a customer-facing chatbot:
| OWASP risk | What it looks like in a business chatbot | Control |
|---|---|---|
| LLM01 Prompt injection | A customer talks the bot into ignoring its rules | Enforce rules in code; treat input as untrusted |
| LLM02 Sensitive information disclosure | The bot reveals another customer’s data or internal notes | Server-side permission checks; separate knowledge stores |
| LLM05 Improper output handling | Model output is passed to a system without checks | Validate and sanitise before use |
| LLM06 Excessive agency | The bot can refund, cancel or edit more than its job needs | Narrow tools, caps, confirmations |
| LLM09 Misinformation | Confident, wrong answers | Grounding, citations, refusal when unsure |
| LLM10 Unbounded consumption | Abuse runs up model costs or degrades service | Rate limits and spend caps |
Disclaimers are not a strategy
A line saying “this assistant may make mistakes” is honest and worth showing. It is not a defence to rely on. In the Air Canada case, the chatbot even linked to the correct policy, and the airline was still liable for what the chatbot said. A disclaimer cannot turn an unreliable design into a reliable one. Spend the effort on making the bot right, and on catching it when it is not.
When it does go wrong
Have the response ready before you need it:
- Deal with the customer fairly and quickly. Honouring a small wrong promise is often cheaper than disputing it, and a decision-maker may well hold you to it anyway.
- Find the cause in the logs — missing knowledge, wrong retrieval, a tool error, or manipulation.
- Fix the source, not just the answer: update the knowledge, tighten the rule, or narrow the tool.
- Add the case to your test set so the same failure is caught before the next release.
- Check whether personal data was exposed. If so, breach procedures apply, including notification within 72 hours where customers’ rights are threatened.
Test before customers do
Before launch, and before every significant change, test deliberately:
- a set of real questions with agreed answers, in Arabic and English;
- the edge cases of every policy — exceptions, deadlines, partial refunds;
- adversarial attempts: requests to ignore instructions, to reveal other customers’ data, to apply discounts that do not exist;
- questions the bot should refuse, and questions it should hand over.
Then review a sample of live conversations every week. What the reviews find becomes next week’s tests.
Frequently asked questions
Is a business responsible for what its chatbot tells customers? The clearest public decision, Moffatt v. Air Canada in 2024, held the airline liable for negligent misrepresentation by its website chatbot. The tribunal rejected the idea that the chatbot was responsible for itself. In Oman, the Electronic Transactions Law attributes documents issued through your automated agent to you.
Can a chatbot create a binding contract in Oman? Yes. Article 19 of the Electronic Transactions Law permits contracts between an automated agent and a person who knows, or is presumed to know, that they are dealing with one. Design order and booking flows with clear terms and an explicit confirmation step.
Does a disclaimer protect us if the chatbot is wrong? Do not rely on one. In the Air Canada case the chatbot linked to the correct policy and the airline was still held responsible for the chatbot’s statement. Disclaimers are worth showing for honesty, but reliability has to come from design.
What is a chatbot hallucination? A fluent, confident answer that is not supported by your sources — an invented policy, price or fact. It commonly happens when the bot finds no relevant information and fills the gap. Grounding answers in retrieved sources and refusing when nothing is found are the main controls.
What should we do when our chatbot gives a wrong answer? Resolve it fairly with the customer, find the cause in the logs, fix the knowledge, rule or tool that caused it, and add the case to your test set. If personal data was exposed, follow your breach procedure.
Do customers have a right to challenge a chatbot’s decision in Oman? Since 7 September 2026, Article 14 of the Personal Data Protection Law gives people the right to object to decisions resulting from automated processing, and requires a human to review the objected decision.
This article states the position under the laws in force on 21 October 2026, the date of publication. It is not legal advice. The Air Canada decision is a Canadian tribunal decision cited for its reasoning; it has no binding effect in Oman.
Sources: Moffatt v. Air Canada, 2024 BCCRT 149 (decision) · CTV News — Air Canada ordered to pay for its chatbot’s mistake · OWASP — Top 10 for LLM Applications · Royal Decree 39/2025 — Electronic Transactions Law (Arabic text) · Royal Decree 68/2026 amending the Personal Data Protection Law
Want a chatbot that can’t easily promise what you don’t offer? ZAWAT builds AI and automation with grounded answers, validated figures, confirmation steps and human handoff designed in. Book a call.