When AI customer service goes wrong: the party boat.
Our AI employee did almost everything right on its first day: ten emails sorted, a scam caught, a refund handled calmly. Then it promised a customer a party boat. The business does not sell party boats. Here is why that happens, and the two sentences that fixed it.
Short answer: AI assistants go wrong in customer service mostly by being too eager to please: they promise things the business does not offer, or say something is already done when it is not. In our test both happened on the first run. Two plain-English lines in the instructions fixed both, and the second run took 41 seconds.
What happened
The test inbox belonged to Bluebird Travel, a demo travel agency we made up for episode 1. It sells flights, hotels, tours and airport transfers. One of the ten emails came from Emma, planning a hen party in Mykonos. The AI's draft was cheerful, warm and wrong: "we can arrange a party boat."
Nobody had told it Bluebird sells party boats. Nobody had told it the opposite either. So it filled the gap with the answer that would make Emma happiest.
The same run had a second, quieter mistake. A customer asked to cancel an airport pickup, and the draft said "we're arranging the cancellation with the driver now." Nobody was arranging anything: it was a draft waiting for a human.
Why AI customer service makes promises like this
- It is trained to be helpful. When it does not know, "yes, we can" sounds more helpful than "let me check".
- It cannot see your business. It only knows what is in the instructions. A gap in the instructions is an invitation to guess.
- It writes like a person who already acted. "We're doing it now" reads naturally, even when the AI has no way to do anything.
None of this is rare or specific to one tool. It is the most common way an AI employee lets you down, and it is also the easiest to fix.
The fix: two lines
We added two sentences to the instruction page:
- Only promise what we actually sell: flights, hotels, tours,
airport transfers. For anything else (boats, restaurants,
events), say we'll check and come back.
- Never say an action is already happening ("we're cancelling
it now"). Say we've received the request and will confirm.Second run, 41 seconds. Emma's draft now said: "we don't book party boats ourselves, but we'll look into it." The pickup draft said the request was received and would be confirmed. Everything else stayed as good as before.
How to catch these mistakes in your own AI
- Read the drafts for promise words. "We can arrange", "we'll refund", "it's done", "guaranteed". Each one should match something you really offer.
- Give it a sell list. One line with exactly what you sell. It is the line that fixed the party boat.
- Separate "received" from "done". The AI can confirm it got a request. Only a person confirms it happened.
- Keep a human on send. In our test the AI sent nothing. That is why a wrong promise stayed a draft and never reached a customer.
- Turn every mistake into a line. The instruction page grows with each test. Ours started with six rules and needed two more after the first run.
The full instruction page, with both fixes in it, is in the customer service prompt guide and in the free kit.
Questions
Why does AI make things up in customer service?
Because it is built to be helpful and it only knows what you tell it. When the instructions leave a gap, it fills it with the friendliest answer, even if that answer is not true for your business.
How do I stop an AI from promising things we do not offer?
Give it an exact list of what you sell and a rule to say you will check for anything else. In our test that single line stopped the party boat promise.
Should AI answer customers without a human?
For a small business we recommend drafts approved by a person. In our test the AI made two wrong promises on the first run, and neither reached a customer because a human approves every reply.