What AI Can and Cannot Do (Yet)
A capability map: what current AI is genuinely good at, what it fails at, and the dangerous middle category where it looks competent but isn't.
On this page
Most disappointment with AI tools comes from using them for the wrong category of task. The capability boundary is real, and it is not where intuition puts it.
Here is the map, in three parts. The third part is the one that costs people money.
Genuinely good
Transforming text you provide. Summarizing, reformatting, translating, changing register, extracting structure from prose. The source material is in front of it, so the failure modes are mild.
Producing first drafts. Code, emails, documentation, test cases. Quality varies, but “edit this” beats “start this” almost every time.
Explaining at an adjustable level. The same concept re-pitched as many times as you need, in whatever framing helps.
Bridging vocabulary gaps. You describe a thing badly; it names the thing. Enormously useful when entering an unfamiliar field.
Bulk classification of messy input. Sorting thousands of tickets, tagging sentiment, routing messages. Individually trivial, collectively expensive for humans.
Working code in well-trodden territory. Common patterns in popular languages, where a great deal of similar code existed in training.
Genuinely bad
Arithmetic on specific numbers. Not because math is hard, but because it is predicting digits as tokens rather than calculating. Give it a calculator via tool calling instead.
Character-level operations. Counting letters, reversing strings, precise character positions. The tokenization resolution problem.
Anything past its training cutoff. No knowledge of last week. It cannot tell you it does not know, either, which compounds the problem.
Knowing what it does not know. Hallucination is not a bug to be patched; it follows directly from the generation mechanism.
Reliable citations. It generates text shaped like a citation. Plausible-looking references to papers that do not exist are a well-documented failure.
Genuinely novel reasoning far from training data. It interpolates impressively within familiar territory and degrades outside it.
The dangerous middle
This is the category worth internalizing, because output quality gives no signal about correctness.
Legal, medical, financial specifics. Fluent, professional-sounding, occasionally wrong in ways that matter.
Code that runs but is subtly wrong. Off-by-one errors, missed edge cases, plausible-looking security holes. It compiles, tests pass, and the bug surfaces in production.
Summaries that drop the crucial caveat. The summary reads well. The one sentence that reversed the conclusion is gone.
Confident historical or technical detail. Dates, attributions, version numbers — delivered with identical confidence whether accurate or invented.
The pattern: fluency is uniform, accuracy is not. The model was optimized for the first, and there is no visible marker separating a well-grounded claim from a fabricated one.
The one practical rule
Ask: would I notice if this were wrong?
If you would notice immediately — code that fails to run, a translation you can spot-check, a draft you will rewrite anyway — use the tool freely. The error surface is visible.
If you would not notice — an unfamiliar domain, a long document you will not read closely, a number you have no way to sanity-check — you need verification outside the model, or you should not be using it for that task.
What will and will not change
Improving fast: reasoning depth, code quality, longer context handling, tool use reducing arithmetic and lookup failures. RAG meaningfully reduces the knowledge-cutoff problem.
Structural, not going away soon: no reliable self-assessment of confidence, no guarantee against fluent fabrication, and the fact that character-level work fights the input representation.
What to remember
- Strong at transforming text you supply, drafting, explaining, and bulk classification.
- Weak at arithmetic, character-level operations, recency, and knowing its own limits.
- The dangerous middle is where output looks expert and may not be — fluency is uniform, accuracy is not.
- Practical filter: would I notice if this were wrong?