Reports show Pangram is strong at AI text detection, not proven for images

In short: Pangram is widely reported to be accurate at spotting AI written text, but its claimed accuracy for AI generated images is not backed by similar independent testing in the same coverage.

What's going on

Pangram is an AI detector, which means it tries to tell whether something was made by a chatbot or by a person. Recent reporting says Pangram performs very well when the input is text. It is described as especially good at telling human writing apart from chatbot generated writing.

Pangram says its text detector can identify AI generated text with about 99.98% to 99.99% accuracy. It also claims a 1 in 10,000 false positive rate for human text, meaning it would rarely label a real person’s writing as AI. Independent coverage also points to a University of Chicago study that found near zero false positives on longer writing samples, and that Pangram did better than competing text detectors in those tests.

But the same reporting also highlights limits. Pangram appears less reliable when a piece of writing mixes human and AI sentences, when the sample is short, or when the text has been heavily rewritten. One report notes it may flag a mostly human document as AI if a few AI generated sentences are inserted, like adding a few printed lines into a handwritten letter.

On images, the situation is less clear. One secondary report says Pangram announced an AI image detection model with 99.5% accuracy, but the coverage does not show comparable independent benchmarks for images.

What to watch

If schools, employers, or publishers use detectors like Pangram, the key question is how they handle edge cases, especially mixed writing and short samples. For images, look for third party testing that shows how often the tool is right or wrong before treating results as reliable.

Source: NYTimes