How an AI detector decides a sentence is 'AI' — and why it misfires

Every week someone asks us a version of the same question: "The detector says my essay is 62% AI. I wrote it myself. What is it seeing?"

The honest answer is that it is not seeing you at all. No detector, ours included, has any information about who typed the words. What it has is the words, and a model of how likely each one was given the words before it. This article explains that mechanism in plain terms, because once you understand it, the score stops being a verdict and becomes what it actually is: a measurement of how predictable a passage is.

Predictability is the whole game

Large language models write by choosing, at every step, a likely next word. Left to their defaults they choose the safe option a lot. The result is prose where each word is roughly what a statistical model would have guessed — smooth, evenly paced, low on surprises.

Human writing is lumpier. We start a sentence one way and swerve. We use a specific word where a general one would do. We write one nine-word sentence and then a forty-word one. Detection research gave these two qualities names: perplexity, roughly how surprised a language model is by each word, and burstiness, how much that surprise varies across a passage. Low perplexity and low burstiness look machine-made. High and uneven look human.

A detector runs your text through a language model of its own, measures those quantities (and, in modern systems, many subtler features learned from labelled examples), and reports how strongly the pattern resembles machine output. That resemblance is the number you see.

Notice what is missing from that description: any knowledge of your intent, your process, or your identity. The score is about the text's statistics, full stop.

Why our detector highlights sentences

Because the signal is statistical, it varies within a passage. A paragraph can be mostly unremarkable human prose with two sentences that happen to be extremely predictable — a stock transition, a textbook definition, a list of common examples in the common order.

That is why Kinzo's AI Detector returns not just an overall AI score and a human score but a set of highlighted sentences. The highlights are the lines that contribute most to the AI reading. In practice this changes how people use the result. Instead of arguing with a percentage, they look at the three sentences that drove it, and usually one of two things is true: the sentence really was pasted from a model, or it is a genuinely bland sentence that could be sharpened whether or not a machine wrote it.

The four situations where detectors misfire

We publish these because a tool that only tells you when to trust it is only half useful.

Short text

Statistics need samples. A fifty-word paragraph does not contain enough words for perplexity and burstiness to mean much, so the score on short input swings wildly. We would put little weight on anything under a couple of hundred words. A few hundred words is a sensible minimum.

Formulaic genres

Some writing is supposed to be predictable. Lab reports follow a template. Legal boilerplate is boilerplate by design. A cover letter that opens with "I am writing to apply for the position of…" is following a convention every human applicant follows too. Text like this reads as low-perplexity because the genre demands it, not because a model produced it.

Non-native English writers

This one matters most and is the least talked about. Writers working in a second language tend, quite reasonably, to use vocabulary they are sure of and sentence shapes they know are correct. That produces lower perplexity — and detectors punish it. A 2023 study by researchers at Stanford (Liang and colleagues, published in the journal Patterns) found that several widely used GPT detectors flagged a majority of essays written by non-native English speakers as AI-generated, while classifying essays by native speakers correctly almost every time. When the same essays were rewritten with more varied vocabulary, most of them stopped being flagged.

If that describes you, a high score on your own writing is not evidence of anything except that you write carefully in a language you learned. It is also why we think the Humanizer has a legitimate use for human writers: varying rhythm and word choice is good editing advice regardless of who wrote the draft.

Edited machine text

The reverse failure. Text that started as a model's output and was then reworked by a person — sentences reordered, specifics added, a few paragraphs rewritten — often lands in the ambiguous middle. Detectors are much better at raw model output than at collaborative documents, which are increasingly what real documents are.

What the big players concluded

It is worth knowing that OpenAI released its own AI-text classifier in early 2023 and withdrew it that July, saying publicly that its accuracy was too low. The company that built the models decided it could not reliably detect them. That does not make detection useless — the technology has improved, and sentence-level signals are more informative than a single score — but it should calibrate expectations. Anyone selling a detector as proof is selling something the field has not delivered.

How to actually read a score

Here is the process we suggest, whether you are checking your own draft or someone else's submission.

1. Check the length first. Under a couple of hundred words, treat the number as noise.
2. Read the highlighted sentences, not the percentage. Ask of each one: is this a sentence a person would write in this document? Often the answer is "yes, but it is a boring sentence," which is a different problem.
3. Consider the genre. Templates, boilerplate and conventional openings will read as predictable because they are.
4. Consider the writer. If the author is writing in a second language, discount the score heavily.
5. Never act on the score alone. For a teacher or editor, the right next step after a high score is a conversation, a draft history, or a request to talk through the argument — not a penalty.

For your own drafts, the score is most useful as an editing prompt. If the detector flags a sentence and you agree it is flat, fix it: add the specific detail, break the rhythm, say the thing you actually mean instead of the thing that sounds like an essay. The sentence gets better and the score follows.

What Kinzo does and does not claim

Kinzo's detector reports a probability. It does not report a fact. We say this on the tool page and in the help center and we are saying it again here, because the cost of getting it wrong is borne by the person being judged, not by the tool. Checking revisions, sentence by sentence, is the use we think detectors are genuinely good for.