AI image detectors are unreliable in both directions. Independent testing has found them declaring authentic photographs fake at rates of up to 40 per cent, while missing heavily manipulated images entirely. The reason is not poor engineering. It is that a detector’s output is a probability, and probabilities behave badly when the thing being looked for is rare.
How accurate are AI image detectors?
Far less accurate than their marketing suggests, and inconsistently so. A NewsGuard audit published in May 2026 ran 15 authentic news photographs, plus lightly and heavily edited versions of each, through five leading detectors. The spread of results was extreme.
ScamAI labelled six of the 15 genuine photographs as AI-generated, a 40 per cent false-positive rate. ZeroGPT flagged three. Hive and Sightengine flagged none at all.
The tools that avoided false alarms then failed the opposite test. Sightengine missed 10 of the 15 significantly manipulated images, and Hive missed six. No single tool in the audit was both cautious about real photographs and sensitive to altered ones.
| Detector | Real photos wrongly flagged | Manipulated images missed |
|---|---|---|
| ScamAI | 6 of 15 | 3 of 15 |
| ZeroGPT | 3 of 15 | 1 of 15 |
| AI or Not | 1 of 15 | 0 of 15 |
| Hive | 0 of 15 | 6 of 15 |
Why do detectors call real photographs fake?
Because they are classifiers trained on artefacts, not on truth. A detector learns the statistical fingerprints left by the generators in its training set, then scores new images for similarity to those fingerprints. Anything that reshapes those statistics, such as compression, resizing, denoising or a phone’s computational photography pipeline, moves a genuine image towards the fake side of the boundary.
Modern smartphone photographs are heavily processed before they are ever saved. Multi-frame stacking, machine-learning noise reduction and sharpening all leave traces that resemble generated content, because the underlying techniques overlap. The same is true of anything that has been through an upscaler, which invents plausible detail rather than recovering it.
The same mechanism explains the misses. A detector trained on last year’s diffusion model outputs degrades sharply on this year’s, and a heavy edit can erase the artefacts it was looking for. Accuracy figures quoted from clean, freshly exported test images do not survive contact with the open web.
What does the statistics problem actually mean?
It means a positive result can be mostly wrong even when the detector is mostly right. When AI-generated images are a small fraction of what is being tested, the small percentage of real images misclassified outnumbers the true detections, and the tool’s positives become worthless as evidence.
A peer-reviewed study in PeerJ, published February 2025, put numbers to this. The authors ran 96 western blot images, 48 generated and 48 taken from 2015 papers, through three detectors. Illuminarty produced 28 false positives out of 48; Is It AI produced 22.
Their conclusion is the important part. At realistic rates of AI-generated images in the literature, the positive predictive value fell to roughly 0.01, meaning about one in a hundred positive results would be correct. The authors wrote that concluding an image is false “based on a high AI probability given by an AI detector would often be misleading”.
A detector that is 95 per cent accurate on a balanced test set can still be wrong on ninety-nine of every hundred images it accuses, once the real world stops being balanced.
Why is a percentage score misleading?
Because it reads like a measurement and behaves like a guess. A detector reporting 92 per cent AI is not saying there is a 92 per cent chance the image is generated. It is reporting how far the image sits on one side of a decision boundary, on a scale the vendor chose.
Those numbers are not comparable between tools, and they are not calibrated against real-world prevalence. Two detectors can return 90 per cent and 10 per cent on the same file without either being broken.
The interface encourages the wrong reading. A confident percentage on a clean dashboard invites people to treat the result as a finding, which is exactly how false accusations happen, against photographers, illustrators and students alike.
What works better than a detector?
Evidence about the file’s origin, rather than inference from its pixels. Provenance metadata, the original capture file, and the publication history of the image are all checkable in ways a classifier score is not, and they fail loudly rather than quietly.
- Content Credentials. A signed C2PA manifest records what made the file and what edited it. Present and valid is strong evidence; absent proves nothing.
- The original file. A raw or unedited camera file with intact photo metadata is far harder to fake than a re-exported JPEG.
- Reverse image search. TinEye and Google Lens establish where an image appeared first, which settles many disputes outright.
- The source. Who published it, when, and whether they can produce the surrounding frames.
None of these is a one-click answer, which is precisely why detectors remain popular. Convenience is the feature being sold, and the accuracy claims exist to justify it. Working through the manual checks on a single suspect file is slower, and it is the only approach that produces an answer worth defending. There is a longer walkthrough of those checks in this guide to the free AI image tools and what each one leaves behind.
When is a detector still worth running?
As a triage signal on large volumes, never as a verdict on one image. A detector that flags a subset of a thousand uploads for human review is doing useful work, because the cost of a false positive is a second look rather than an accusation.
Anyone publishing images made with free generative tools should also expect their own work to be flagged inconsistently by these services. The failure mode is using a single score to make a decision about a single person. Running two or three detectors and treating disagreement as a stop signal is a reasonable middle path, given how far apart their results were in the NewsGuard audit.
The bottom line
AI image detectors measure resemblance to the generators they were trained on, not authenticity. Published audits show them wrongly flagging up to 40 per cent of genuine photographs, and a peer-reviewed study puts the chance that a positive result is correct at roughly one in a hundred under realistic conditions.
Treat any score as a prompt to look further, never as proof. Where the answer matters, the file’s provenance and publication history will settle it and the percentage will not.
Frequently asked questions
Are AI image detectors accurate?
Not reliably. NewsGuard’s May 2026 audit found one detector labelling 40 per cent of authentic news photographs as AI-generated, while another missed 10 of 15 heavily manipulated images. Accuracy varies by tool, by generator and by how much the file has been edited.
Why do detectors flag my own photos as AI?
Because modern phone cameras apply machine-learning noise reduction, multi-frame stacking and sharpening before saving. Those processes leave statistical traces that resemble generated content, and a classifier trained on artefacts cannot tell the difference.
Does a high percentage score mean an image is definitely AI?
No. The percentage reports distance from a decision boundary, not probability. Scores are not calibrated to real-world prevalence and are not comparable between tools, so two detectors can return opposite results on the same file.
Is there a detector that works properly?
None tested well in both directions. Tools that avoided false positives in the NewsGuard audit were the ones that missed the most manipulation, which is the expected trade-off when a threshold is tuned rather than a genuine accuracy gain.
What should be used instead?
Provenance evidence. Content Credentials, the original unedited capture file, reverse image search and the publication history all point at where a file came from, which is a checkable question rather than an inference from pixels.
