Detector websites promise remarkable accuracy. Studies find detectors that wrongly flag more than half of the essays in a test group. Both can be true at the same time. It depends on which texts were tested, which errors were counted and where the threshold was set.
This is part 4 of Level 2 in our AI Security learning path. Level 0 asked whether AI-generated content can be detected at all, and our article on how our AI text detector works explains the nine signals of our own tool. This part goes one level deeper: how the four approaches work inside, how to read accuracy claims, why paraphrasing and base rates break detectors, and how schools and HR teams can build a fair process around them.
Four approaches and how they work inside
Tools that answer “AI or not?” rely on four approaches, sometimes in combination. The first two guess after the fact. The last two need the cooperation of whoever generated the content.
Statistical tests measure properties of the content itself. For text, the key measure is perplexity: how surprising each word is for a language model. AI text tends to be predictable, so it scores low. Other signals are the variation in sentence length and repeated phrases. These tests need no training on examples, and their logic is easy to explain. Their weak spot: plain, formal or simple human writing is also predictable. Our own detector belongs to this group. It scores texts with statistical signals right in your browser:
Trained classifiers are models that have learned from many labeled examples of human and AI content to tell the two apart. Most commercial detectors for text, images and audio work this way, such as Pangram Labs for text, which according to the vendor is aimed at educators, publishers and organizations. Classifiers can learn subtle patterns. But they are only as good as their training data: content from a new generator, a different language or an unfamiliar genre can look different from everything they have seen.
Watermarks are hidden signals that a generator puts into its own output, and the next section shows how this works for text. Content credentials are signed metadata that record who or what made a file and with which tool. They follow the C2PA standard of the Coalition for Content Provenance and Authenticity. Adobe, for example, says it attaches them to images fully generated with Firefly. Both methods give strong evidence when present, but only for content from tools that take part.
| Approach | Needs the generator’s help? | Typical media | Main weakness |
|---|---|---|---|
| Statistical tests | No | Mainly text | False alarms on plain or formal writing |
| Trained classifiers | No | Text, images, audio, video | Weaker on new generators and unfamiliar content |
| Watermarks | Yes | Text, images, audio, video | Only for participating tools; heavy edits weaken them |
| Content credentials | Yes | Images, video, audio | Easy to strip; a missing credential proves nothing |
How a text watermark works
A text watermark hides in word choice. While the model writes, a secret key marks part of the possible next words as “preferred” at every step. The model gives these words a small extra push, small enough that the text still reads naturally. A detector that knows the key counts the preferred words. In human text, only the share you would expect by chance is preferred. In watermarked text, it is clearly more. A statistical test turns the count into a confidence, and longer texts give more certainty. Researchers published this idea in 2023 (Source: Kirchenbauer et al., “A Watermark for Large Language Models”, ICML 2023). Google later published its own method for text, SynthID Text, as open source.
The principle also explains the limits. Rewriting replaces many of the preferred words, and the count drifts back toward chance. Google says that the confidence of its text detector drops considerably when a text is thoroughly rewritten or translated. According to Google, the watermark also works less well for factual answers, where there is little room to vary the wording. Only the key holder can check a watermark, so you need the provider’s own detector.
Read accuracy claims like a tester
Every detector makes two kinds of errors. A false positive flags human work as AI, and it can hurt an innocent person. A false negative lets AI content pass. “Accuracy” mixes both into one number and can hide a high false positive rate. Two measures tell you more: the detection rate (how much AI content the tool catches) and the false positive rate (how much human content it wrongly flags), both at the threshold the tool actually uses.
Level 0 described two warnings. OpenAI withdrew its own text classifier in 2023 because of low accuracy. And in a Stanford study, seven detectors flagged, on average, more than half of the essays by non-native English writers as AI. A claim is only worth as much as the test behind it. Before you rely on a detector, ask the vendor:
| Question | Why it matters |
|---|---|
| What is the false positive rate, and at which threshold? | This number decides how many innocent people get flagged |
| Which texts were tested: language, length, genre, writers? | Results on news articles say little about student essays |
| Were non-native writers included? | They are flagged more often by many detectors |
| Which generators and versions were covered? | New models can look different from the training data |
| What does a score of 80 mean? | Only a calibrated score can be read as a probability |
| Where do uploaded texts go? | Essays and applications contain personal data |
Why paraphrasing and editing break detectors
Detectors look for traces of the generation process. Every change after generation can erase them. In one study, an automatic paraphrasing model reduced the detection rate of one research detector from 70.3 to 4.6 percent, at a fixed false positive rate of 1 percent. The same paraphraser also got past watermarks and commercial detectors (Source: Krishna et al., “Paraphrasing evades detectors of AI-generated text, but retrieval is an effective countermeasure”, NeurIPS 2023).
Editing by people has a similar effect. And most real texts are mixed anyway: a person writes a draft and lets AI polish it, or AI writes a draft and a person rewrites half of it. A detector gives one score for the whole text, although “AI or human” is no longer a yes-or-no question. For images and video, compression and screenshots do the same damage to pixel traces and metadata, which is why checking the source and context matters more there. Level 1 shows how to verify images, audio and video step by step.
Base rates: why many flags can hit people
Even a good detector raises many false alarms when most texts are human. The share of AI texts in a group is called the base rate. The rarer AI texts are, the larger the share of human texts among the flags. Level 0 showed this for a single case with assumed numbers. The chart shows the whole curve for an example detector that catches 90 percent of AI texts and wrongly flags 2 percent of human texts:
Calibration is the related question of what a score means. A detector is well calibrated if, of all texts that get a score of 80, about 80 percent really are AI-written. A detector can be calibrated for one population, such as English news texts, and badly calibrated for another, such as essays by students learning English. So read a score as a statement about texts like the ones the detector was tested on, not as the probability for the person in front of you.
A fair process for schools and HR
A detector can start a question, but a person must answer it. Level 0 gave five rules for reading a single score fairly. Schools and HR teams need more: a process that everyone knows in advance. Seven steps make it fair:
- Set the rules first. Say which AI help is allowed. In many classes and jobs, AI for spelling or structure is fine.
- Tell people in advance which tools you use, for what purpose and what happens to their texts.
- Use a flag only as a trigger for a closer look, never as a grade or a rejection.
- Check whether the text suits the tool: length, language and type of text.
- Collect process evidence: drafts, version history, notes and sources.
- Talk to the person. Someone who wrote a text can usually explain how it came about.
- Let a person decide, document the reasons and allow an appeal.
In hiring, the bar is even higher. Many applicants polish their cover letters with AI, so a detector says little about their skills. In the EU, people have the right not to be subject to a decision based solely on automated processing that significantly affects them (GDPR Article 22). Rejecting an applicant because of a detector score alone is exactly the kind of decision to avoid.