AI tools predict likely text. The biggest risk that comes from that: answers that are confident, well written, and wrong.
Why AI makes things up
When an AI tool does not have a good answer, it rarely says so on its own. It keeps doing what it always does: producing text that looks like a good answer. People call this a "hallucination." In plain words, it means the AI made something up and presented it as fact.
This is not the tool lying. It has no idea what is true. It only knows what a believable answer tends to look like. A believable answer to "What do I need to renew my contractor's license?" includes a form number, a deadline and a fee. So the AI may give you all three, even if it has to invent them.
People use tone as a clue. When a coworker says "I think it's Tuesday, but let me check," you know to check. AI tools usually write in the same smooth, confident voice whether the answer is solid or invented. There is no hesitation in the wording to warn you.
That is why polish is not proof. A neatly formatted answer with headings and bullet points can still be wrong in every detail that matters.
Four common slip-ups
Watch for these in anything AI gives you.
1. Invented sources, quotes and laws. AI can produce a court case that never happened, a quote nobody said, a regulation number that does not exist, or a study with a realistic title and no real author. If an answer names a source, assume you need to find that source yourself before you rely on it.
2. Out-of-date knowledge. Most AI models learn from text gathered up to a certain point. After that, they know nothing new unless they are connected to a live search. Tax rules, insurance requirements, prices and contact details change. The AI may give you an old answer without saying so.
3. Math slips. Because it predicts text rather than calculating, AI can add a column wrong, misplace a decimal, or mix up a percentage. It may show its work neatly and still get the total wrong. Any number that matters gets checked with a calculator or your own spreadsheet.
4. Confident details about your world. Ask about a local supplier, a client, or your own company, and the AI may fill gaps with guesses: a wrong address, a made-up phone number, a job title that sounds right.
What this looks like at work
Picture a bookkeeper who asks an AI tool which expenses a small client can deduct. The answer is clear and organized, and it cites a rule by number. It reads well. But the rule number is made up, and one item on the list does not apply to that kind of business. If the bookkeeper copies it into a client email, the mistake now carries the firm's name.
Or a property manager asks how much notice a tenant must get before a lease ends. The AI gives a specific number of days. It may be right for one state and wrong for theirs, or simply invented. Anything legal, medical or financial needs to go to a qualified professional, not straight from AI to a customer.
Simple habits that help
You do not need to stop using AI. You need to read its answers the right way.
- Treat every specific fact (names, numbers, dates, sources) as "unconfirmed" until you check it.
- Tell the AI it may say "I don't know" when it is unsure. This helps, though it does not fix the problem completely. Our guide to a good prompt shows how.
- Ask where a fact comes from, then find that source yourself.
- Use AI for the shape and wording of a document, and your own records for the facts.
Remember this: the more specific and important a detail is, the more it needs a human check.
Key points
- AI can make up facts, sources, quotes and laws and present them confidently. This is called a hallucination.
- A confident, polished answer is not proof that it is correct.
- AI knowledge can be out of date, and it can make math mistakes.
- Check every specific detail that matters, and send legal, medical or financial questions to a qualified professional.
Spot a made-up answer
Use this whenever an AI answer includes specific facts you might rely on.
- Does it name a law, rule, study, quote or court case? Find the original yourself.
- Does it give a number, total or percentage? Recompute it with a calculator or spreadsheet.
- Could the answer have changed recently (prices, rules, contacts)? Check a current official source.
- Does it include a name, address or phone number you didn't give it? Check your own records.
- Does it sound sure? Remember that confidence is not proof.
- Is it legal, medical or financial? Send it to a qualified professional before use.