AI chatbot giving wrong answers? How to find the cause and fix it

First, collect the wrong answers

Do not start by changing the prompt. Export the conversation logs and pull out twenty answers that were wrong. For each one, note the question, the answer, and what the correct answer was and where it is written down. If your chatbot does not keep logs, that is the first fix, because nothing below can be diagnosed without them.

Then sort the twenty by cause, using the table below. Most bots have one dominant cause.

The six causes, with before-and-after examples

The examples below are illustrations of each cause, not transcripts.

1. No grounding: the bot answers from the model's general knowledge

Before. "Do you deliver to Shillong?" "Yes, we deliver across India within 5–7 days." Nobody told it that; it filled the gap with a typical answer. OWASP's 2025 list of risks for LLM applications describes this directly: hallucinations happen when models "fill gaps in their training data using statistical patterns".

After. The bot looks up your serviceable-pincode list before answering and says "I can't confirm delivery to that pincode. Here is how to check" when it is not listed.

Fix. Connect the bot to your own content and data, and instruct it to answer only from what it retrieves. This is what retrieval-augmented generation does.

2. Wrong passage retrieved

Before. "What is the warranty on the 1.5-ton model?" The bot answers with the warranty for the 1-ton model, because both passages look similar and the search picked the wrong one.

After. Each document chunk carries its product, model number and date as metadata, and the search filters on them before ranking.

Fix. Check what was retrieved for each wrong answer. If the right passage was not in the retrieved set, the problem is retrieval, not the model: chunking, metadata and search settings need work. This is most of the job in RAG development.

3. Stale source documents

Before. The bot quotes last season's price list, because the old PDF and the new one are both in its knowledge base.

After. One current version per document, an owner for each, and a dated re-index whenever a price or policy changes.

Fix. Remove superseded files, and make "update the bot" a step in the process that changes a price or policy.

4. Instructions that allow guessing

Before. "Can I return a used item?" The bot, told to "always be helpful", says yes.

After. The instructions say: answer only from the provided documents; if they do not cover the question, say so and offer a person. Anthropic's guidance lists giving the model explicit permission to say "I don't know" as a basic step, along with restricting it to the provided documents and checking each claim against a quote.

Fix. Rewrite the instructions to prefer "I don't know" over a guess, and require the bot to cite the passage it used.

5. No test set

Before. Each complaint leads to a prompt change, which fixes that answer and quietly breaks two others.

After. A list of fifty to a hundred real questions with correct answers, run after every change. A change ships only if the score does not drop.

Fix. Build the test set from your logs. It is the single most useful thing you can own about your bot.

6. No handoff

Before. An angry customer asking for a refund gets a policy paragraph, then another.

After. Refunds, complaints, legal and medical questions, and any conversation where the customer asks twice for a person go to staff with the conversation attached.

Fix. Define the topics the bot must not answer and build the route to a person. The design is covered in stopping an AI chatbot from making things up.

Why this matters more than it looks

A wrong answer from a bot on your site is your answer. In Moffatt v. Air Canada (February 2024), British Columbia's Civil Resolution Tribunal heard that the airline's chatbot had told a customer they could apply for a bereavement fare retroactively, which the airline did not permit. Air Canada argued it could not be held liable for information provided by its agents or representatives, including a chatbot. The tribunal called this "a remarkable submission", found that "it should be obvious to Air Canada that it is responsible for all the information on its website", and ordered it to pay C$812.02 in damages, interest and tribunal fees.

When to get help

Fix it yourself if the cause is stale documents or loose instructions; both are content work. Get help when the logs show retrieval failures, when the bot needs live data such as order status, or when you need a test set and handoff built into the system. WavX builds and repairs AI chatbots and starts a repair by reading your logs, not by rewriting the prompt.

If the bot handles a few questions a week, consider switching it off and showing a well-organised FAQ with a contact form. A bot that is wrong one time in ten does more harm than no bot.