When AI sounds right but is wrong

AI-assisted output is now showing up through Copilot in Microsoft 365, Gemini in Google Workspace, embedded summarizers in PDF readers, CRM and inbox tools, and assistants built into accounting and customer-support platforms. The practical question for the business is whether anything checks AI output before it leaves the building, because in many small companies the same staff member who drafts is also the one who reviews the work. A single wrong client deliverable can cost a relationship, and a wrong number in a regulated or contractual context can be disproportionately expensive relative to a small business's revenue and client base. The exposure comes from unverified AI output being used as if it were already checked, which is something the business can address through process even though the underlying AI behavior cannot be eliminated.

Staff should be watching for six recognizable error categories that all share the trait of looking like normal, useful writing. The first category is invented sources: citations, report titles, authors, dates, or quotations that read as real but do not exist, because the AI produces a citation-shaped sentence when the prompt asks for one. The second is stale information presented as current, such as policy, tax, pricing, regulatory, or product information that was true at some point and is now wrong. The remaining categories are content from a different client or project mixed into the current draft, summaries that omit the qualifier or caveat that changes the conclusion, customer-facing AI answers that sound confident but are wrong, and numbers or comparisons that look reasonable but are not anchored to a verifiable source.

The right amount of verification is proportional to the cost of being wrong on a specific deliverable, which varies far more than the volume of AI use does. A rough internal brainstorm or a first-pass draft for the staff member's own use needs light verification because nothing leaves the business yet. A client deliverable, sales document, or any output that informs a real decision needs the specific claims that drive the recommendation checked against the actual source. Regulated work, legal-context summaries, financial recommendations, and customer-facing answers need verification against the authoritative source before the work leaves the building, because the cost of being wrong falls entirely on the business and its relationships.

Checking a citation or number means opening the source the AI named and confirming the claim is there in the form the AI presented it. The four concrete checks are: confirm the source exists at the URL or in the database the AI cited, confirm the title and authorship match what the AI said, confirm the specific number or quote appears in that source, and confirm the date is current enough for the use the business is making of it. A common shortcut staff fall into is Googling the title and accepting whatever a search snippet says, which is not source-checking, because AI sometimes invents titles that produce real-looking search hits at unrelated sources. The honest check is clicking through to the actual document the AI named and confirming the claim is supported in the text itself.

Cross-client mixing happens when AI carries wording, context, numbers, or confidential details from one client's session into another client's draft, usually because the same conversation, project, or memory feature is being reused across clients. The practical separations are: keep a fresh AI session for each client rather than continuing one long session across multiple accounts; use separate AI projects or workspaces for each client where the tool supports it; and turn off memory or persistent-context features for any AI tool used across multiple clients unless there is a reviewed reason to keep them on. This failure mode is the output-side counterpart to confidential data entering AI, with the same confidentiality exposure for the business and the affected clients. Where AI-assisted drafts include client-specific names, dates, dollar figures, or strategy details, the reviewer should confirm those details belong to the client the draft is being delivered to.

Customer-facing AI carries the same wrong-output risk as internal AI use, but the answer reaches a customer in real time without any staff member reviewing it first. The difference from vendor AI features is the focus: that topic covers what it means for the business that a customer-facing AI feature exists at all, while this question covers what to do about it being confidently wrong when it answers a real customer. The highest-leverage control is restricting the AI to approved knowledge sources (the business's own documentation, pricing, policy, and support knowledge base) instead of allowing it to answer from the general model, because answers grounded in the business's actual knowledge are the ones the business can stand behind. The realistic SMB control set is approved-source restriction, an escalation path to a person, clear labeling that the customer is interacting with AI, and answer limits on commitments the business cannot honour.

Whether a specific piece of AI-generated content is legally usable in commercial work is a question the business should take to its own counsel, because it depends on the source material, the AI vendor's terms, the use, the industry, and the jurisdiction. The operational discipline the business can put in place independently is provenance tracking: keeping a simple record of what was AI-generated, what prompt or source the AI was given, what staff edited or rewrote, what review the work received, and what was published or sent commercially. The defensibility question matters because if a customer, regulator, or rights-holder later asks how the material was made, the business should be able to answer by pulling records of how the work was created. Practical questions to take to counsel include which AI vendor terms apply to the company's use, what rights, restrictions, or indemnities the terms provide for business use, what disclosure obligations exist in the firm's industry, and what review standard to require before AI-generated work appears in commercial deliverables.

The qualified reviewer is the person on staff who knows the subject matter of the output. For an AI-drafted report on insurance coverage, the person who knows the firm's insurance practice is the qualified reviewer; for an AI-drafted summary of a contract clause, the person who knows the contract and the surrounding business context is the qualified reviewer. The reviewer should be able to answer three questions about the output before it leaves the business: which claims in this material actually matter, which source proves each of those claims, and what happens if any of those claims is wrong. The practical constraint for small businesses is often making time and applying the framing to actually open the sources and confirm the claims, because the cost of skipping that step shows up externally after the work has already been delivered.

Discovery is partial-coverage by design, because most small businesses do not have document-management systems that tag which deliverables were AI-assisted. The practical paths are: review the high-stakes deliverables by date range starting from when staff began using AI heavily; ask staff directly to flag past work they used AI on, framed as a process review rather than a personal audit; and search the document corpus for known phrases, citation formats, or recurring wording from commonly used prompts where the corpus is small enough to do that practically. Citations, statistics, and named sources in past deliverables are the most checkable items because each one is either real or invented, and a sample check across the highest-stakes work surfaces whether the firm has a broader problem. The goal of the exercise is to find the highest-risk past work that needs re-verification, accepting that discovery will not catch every instance.

The first conversation with the client should acknowledge the issue without speculating about cause, because committing to a cause before checking the disputed claims often turns a fixable error into a credibility problem. Identify the specific claims in the report that are wrong rather than treating the document as a whole as suspect. Pull the source for each disputed claim and confirm directly whether the citation, statistic, summary, or named fact is invented, stale, or correct, because the corrective conversation depends on knowing exactly what failed. Determine whether the same failure exists in other deliverables to the same client and in other AI-assisted work from the same staff member during the same period, because that scope determines what the client needs to know and what other clients may need to be told. If cross-client content was mixed into the report, the response should also treat the original client's confidential information as having reached the wrong audience, which is the same confidentiality exposure covered in confidential data entering AI. Provide a revised version of the report with the corrected claims clearly marked and the sources cited so the client can verify them, and review what changed in the firm's verification process so the same failure does not repeat. Notification decisions for affected clients, customers, or regulators depend on what was delivered and to whom, and stay with the owner and appropriate counsel.