These systems generate plausible text rather than retrieving verified facts, so errors arrive with the same confidence as correct answers. The exposure is largest where a customer relies on the output.

Why the errors look like the truth

The output is generated as plausible continuation rather than retrieved from a checked source. Which means correctness and confidence are not connected.

A human expert hedges when uncertain, and that hedging is how we usually detect the edge of somebody's knowledge. These systems generally do not, so the usual signal is absent.

The practical consequence: you cannot tell from reading it. Tone, specificity and structure are identical whether the content is right or invented.

Which is why the advice is not read carefully but verify independently.

The categories where it goes wrong most

Anything current

Prices, rates, regulations, product specifications, and who holds a position. These change, and training has a cutoff.

Anything jurisdictional

The answer frequently reflects the largest body of writing on a subject, which for most topics is another country. Canadian rules on employment, consumer protection, privacy and tax differ, and a confident answer describing a different jurisdiction is a common failure.

Citations and sources

References that look real, with plausible names, dates and numbers, and do not exist. This has caused documented professional embarrassment in several fields.

Arithmetic in prose

Calculations embedded in text are frequently wrong in ways that read fine, particularly percentages, totals and anything compounding.

Specific local detail

Opening hours, whether a business exists, what a municipality requires, what a supplier stocks. Invented with complete assurance.

What it costs, by situation

The exposure is not evenly distributed.

Low. A draft email with an awkward phrase. You notice, you fix it.

Moderate. A published page with a wrong general statement. Embarrassing, correctable, and possibly a supportability problem if it is a claim.

High. A quote built on a wrong figure, a specification that will be worked to, or advice a customer acts on. Money and reputation, and in a regulated trade potentially more.

Severe. Anything a customer relies on that turns out to be invented, particularly in a field where you hold yourself out as competent. That is a professional problem rather than a technology one.

The framing that matters: nobody accepts drafted by a tool as a defence. What you publish or send is yours.

Verification that is proportionate

Total verification defeats the purpose, so the sensible approach is graduated.

  1. Anything numeric, checked at source.
  2. Anything legal or regulatory, checked against the actual authority.
  3. Any citation or reference, confirmed to exist before repeating it.
  4. Anything about a named business, product or person, confirmed.
  5. Anything you cannot assess, either verified or removed.
  6. General explanation in your own field, read for sense, since you would notice.

The last is the point. Where you have expertise, you are the check, and that is why the tool is more useful to somebody who knows the subject than to somebody who does not.

The inversion worth understanding

It is most valuable where you need it least, and most dangerous where you need it most.

On a subject you know, it drafts quickly and you catch the errors. Real gain, low risk.

On a subject you do not, it produces text you cannot evaluate. Apparent gain, and the risk is exactly what you were hoping to avoid by asking.

Which is uncomfortable, because the second case is the one people reach for it.

Asking it to check itself

Worth a note since it is a common instinct.

Asking whether the output is accurate, or requesting sources afterwards, does not reliably help. The same process generates the reassurance and can generate references for material it invented.

Verification has to be external: the actual regulation, the actual price list, the actual supplier.

Tools that cite live sources are better in this respect and still require following the link, since a real source can be summarised wrongly.

Where a claim ends up on your site, the standard is the same as for any other claim, as set out in claims you can and cannot make.

Frequently asked questions

Why do the errors look like the truth?

The output is generated as plausible continuation rather than retrieved from a checked source, so confidence and correctness are unconnected and the usual hedging signal is absent.

Which categories fail most?

Anything current, anything jurisdictional where the answer reflects another country's rules, citations that do not exist, arithmetic in prose, and specific local detail.

What is the Canadian-specific risk?

Answers frequently reflect the largest body of writing on a subject, which is usually another country, while Canadian employment, consumer, privacy and tax rules differ.

Is drafted by a tool a defence?

No. What you publish or send is yours, and a claim you cannot support is a claim you should not have made regardless of how it was drafted.

What verification is proportionate?

Check anything numeric, legal, cited, or about a named entity at source. In your own field of expertise you are the check, which is why it suits knowledgeable users.

Can I ask it to check itself?

Not reliably. The same process generates the reassurance and can invent references for invented material. Verification has to be external.

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.

Publishing content nobody verified?

We check the factual claims against the actual sources, which is the step that gets skipped because the output reads so confidently.

Start a Conversation