How (Un)Reliable Are AI Detectors? How False Positives Come About and How to Best Deal With Them

Artificial intelligence is causing quite a stir among content publishers who see the value in human-generated content. How can you be sure that the text you ordered really flowed from the pen of a genuine creative writer? Are AI detectors the solution, and how should you interpret the results of these applications? At Textbroker, we’ve done a lot of research into AI detectors. We can provide insight into how these tools work and what role they play in producing content at Textbroker.

How Do AI Detectors Work?

Honestly – when you first used an online tool to detect AI, didn’t you assume that the percentage value you received described how much of the text was created by AI? After all, these sophisticated AI detectors claim to identify text that ChatGPT or similar LLM text generators have written. Unfortunately, that’s not entirely correct. These kinds of tools work with probabilities, so they tell you that the checked text has been created by an AI with a probability of “x” percent.

How does the tool know? The underlying principle is quite simple. Basically, the AI detectors are based on a language model with artificial intelligence, such as ChatGPT. The tools use it to check how strong the probability is that the checked text has been created with the assistance of AI. The AI detector calculates which word is most likely to appear next within the text. Since AI utilizes similar language models, this leads to a high degree of matches in the case of AI-generated text. On the other hand, if the text has been created by a human, the deviations should be so extensive that the AI detector will identify the text as manually created.

Other aspects that the AI detectors check for are spelling and grammatical errors. Textual errors are annoying, but they are human. AI, though, does not make any spelling mistakes. So, a completely error-free text can be indicative of AI-generated text.  Of course, this doesn’t mean an error-free text is automatically an AI product or that errors are automatically proof of human work. AI is no less infallible than a human.

Creativity vs. Probabilities

Two additional factors that AI detectors look for are “perplexity” and “burstiness.” In both cases, a high score indicates that a human must have written the text.

Perplexity

The following examples illustrate the meaning of perplexity in a text:

The first sentence has a highly probable sequel, meaning it’s of lower perplexity. The contents are logically suited, and the second half of the sentence is exactly what a reader would most likely expect. And that is exactly the point: AI very often generates sentences of foreseeable content because AI lacks the creativity with which our authors infuse the text.

Higher perplexity text looks more like the second sentence. There is a turn in the story, which is far removed from the content of the first part. It’s rather improbable that AI would generate such a sentence because artificial intelligence almost always generates plausible sentences with predictable turns.

Burstiness

Burstiness is the metric used by AI detectors to rate variations within sentences. This refers mostly to sentence length and structure. Artificial intelligence tends to construct sentences with similar patterns, acting repetitively. Consequently, the sentences are usually of the same length, and there is little variation. By contrast, humans usually employ a mix of short and long sentences, switching between active and passive voice and using varied sentence structures. Hence, a high burstiness rating would indicate a text generated by a human.

AI Detectors Aren’t Infallible

At first glance, the notion that detector tools could identify AI-generated text seems attractive. Unfortunately, the tools are not infallible. This is mainly because detector tools and technology, as well as AI text generators, continue to evolve. Therefore, AI detector results are not entirely reliable, and regrettably, they often report so-called false positives, results that erroneously classify human-generated text as AI-generated.

There have been several reports of teachers falsely accusing students of turning in AI-generated work due to incorrect detectors. Furthermore, studies have routinely shown that the accuracy of these detectors is spotty and inconsistent. Perhaps you’ve even gotten some false positives after running an AI detector on an article you know you wrote.

Potential Causes of False Positives

Perplexity and burstiness, in particular, can frequently lead to incorrect results. For instance, certain types of content may mandate a certain structure or leave little room for phrasing and creativity. News articles or listicles, for example, have to adhere to a certain type of predictable structure. And company and service descriptions also rely on certain types of content and phrasing.

Texts in which the author must incorporate certain phrases or avoid definitive statements (often for legal reasons) are also prone to false positives. Take healthcare-related texts or financial texts. In these types of documents, authors have to avoid making guarantees. Consequently, they have to utilize predictable phrasing and auxiliary phrases, which are commonly attributed to AI text generators. 

Short texts tend to be problematic as well. They simply do not provide a sufficient number of indicators that would allow the tool to make an objective determination. For this reason, the results of AI detectors should be taken as an indication but never as proof positive. Therefore, the same basic principle applies to AI detector tools as for all other tools: They were created by humans for humans. And they always require a user to verify and interpret the results!

Can you trust your AI detector not to provide false positives?

How Reliable Are AI Detectors Really?

To understand how false positives can come about, let’s look at what kind of information the AI detector results represent. AI tools don’t provide an absolute answer regarding whether a text was generated by AI. Rather, they present you with a probability percentage that indicates the likelihood that a given text stems from a human or from AI. So, a percentage value of 75 does not mean that AI has generated 75% of the text. It merely means that, according to this particular tool, the likelihood that some form of AI was used during the writing process is 75%.

For more on how to interpret the results, see the blog article by Originality.ai on this subject. Additionally, Copyleaks provides a PDF with the most frequently asked questions regarding the AI detector. It provides good examples of what the Copyleaks tool considers AI usage.

The providers of AI detector tools advertise high precision. They frequently claim hit rates of more than 90%. Yet, this means that even the providers themselves don’t consider their own tools to be infallible. False positives can occur. The providers emphasize this in their FAQs and blog articles as you will see further in this blog. Of course, this is very annoying for any author who has taken great care in writing a text without the help of AI.

Our AI Detector Testing: Samples and Results

We put four German sample texts through the detector tools: two category descriptions for a fictitious online coffee trader and two service descriptions for a fictitious locksmith service. One of each pair was authored by a human while the other was written by artificial intelligence. Why did we decide on these particular text types? It has been our experience that text types with specific guidelines for content and wording are more prone to false positives than genres that give authors more room for creativity.

We purposely selected German language texts for our testing. The rationale was that many AI detector tools were originally developed for English. They yield better results in English (see, for instance, the Copyleaks FAQs). We wanted to test, specifically, whether the results are reliable with German texts as well. Additionally, we uploaded our four texts in different entry formats: once as a direct Word file, once via the text entry form on the AI detector’s website, and once with and once without HTML formatting.

Before going any further, let’s be clear about one very important aspect: We tested only a small number of texts, so we neither can nor want to issue any statements about the statistical accuracy of the results that these tools put out. This test is intended to be a demonstration only, and its purpose is to provide food for thought.

Testing Copyleaks

Copyleaks was the first tool that we tested:

The tool marked the first human text as 0% AI three times. Only in the case of the direct entry without HTML was Copyleaks 100% certain that the text was AI-generated, which was wrong. The second text, also generated by a human author, was marked by the Copyleaks AI detector only twice as definitely of human origin. However, the tool is wrong in the case of the text upload without HTML formatting, which it identifies as 100% AI. And the same result came up once more for text number 3 and text number 4 – both of which are entirely AI generated. 

When uploading both types, the HTML formatting is converted to so-called “entities.”  This appears to skew the results. So, here is our first recommendation: If you want to check a text, it’s best to do so without HTML formatting!

Testing Originality.ai

We repeated the exact same testing with Originality.ai, more precisely with the multilanguage model, which is designed for languages other than English:

The tool rated the first human-generated text as AI-generated with a probability of 50% to 51%. The second text (also authored by a human) was even marked as 97% to 100% AI by Originality.ai. Originality.ai correctly identified the third text as 100% AI, independent of the manner in which we performed the upload. In the case of the fourth text, however, it marked the HTML-formatted texts as AI-generated with a probability of only 6%. With the upload without HTML, the result was correct again, at 100% AI.

Statements From the Operators of the Tools

We asked the operators of Copyleaks and Originiality.ai about the accuracy of their tools, under what conditions the tools yield optimum results, what triggers false positives, and whether the use of writing assistants, such as spell checkers and style checkers, can also influence the result. Both replied via email, summarized below.

Copyleaks

According to Copyleaks, its AI detector provides reliable assessments only for texts of a certain minimum length. The minimum length is 350 characters when using the browser add-on and 255 characters with the web app. Copyleaks attributes false positives to the use of additional tools for text optimization, among other factors. For instance, LanguageTool.org offers a function in addition to their spell checker, which allows you to rephrase sentences. This is an AI application, which the AI detector tools will identify as such. However, Copyleaks doesn’t call this a false positive but a correct identification:

”While writing assistant tools have been using AI for a while, many platforms have evolved to use large language models (LLMs) for rewriting portions of content, which can lead to the text being flagged as AI, which, technically speaking, is not a false positive but rather a correct detection of AI content.”

Language is another factor. The detector tool works best with English text. Other languages, such as German, French, and Italian, are supported by the AI detector tool, but the reliability is not yet as high.

Copyleaks emphasizes that their internal language testing on 1,000 English texts didn’t yield any false positives where a spell checker had corrected the text:

“To determine the threshold at which content edited by writing assistant tools gets flagged as AI, we performed a test using two AI-powered writing tools: Grammarly and the Copyleaks Writing Assistant.

One thousand random files from a public essay dataset of English-language text were collected for the test. The dataset is designed to be English only and does not contain AI. The essays were then edited using Copyleaks Writing Assistant and Grammarly. Here are the findings:

One thousand human-created files were run through the Copyleaks Writing Assistant, with each one averaging around 35% of changes made. These updated files were scanned through the Copyleaks AI Detector. All 1,000 came back as human content.” (For more information, see the Copyleaks blog article “Do Writing Assistants Like Grammarly Get Flagged As AI?”)

However, when functionalities for improving sentence structure are applied, the detector tool identifies 31.6% of the texts as AI-generated. Copyleaks lists the following as the main reasons for false positives:

“While the Copyleaks AI Detector has a false positive rate of .2%, there is always the possibility of human-authored text being flagged as AI.  This can occur for several reasons: The content was put through a writing assistant tool using genAI-powered features like GrammarlyGo, which will likely get flagged as AI.  The content was altered by a text spinner or a similar tool. AI was used to create an outline or template.”

Recently, Copyleaks also published a blog clarifying these issues: How Does AI Detection Work? Detailed explanations are also found in their PDF of the most frequently asked questions in regards to the Copyleaks tool.

Originality.ai

The feedback from Originality.ai is similar, referring to their comprehensive Help Center post. Additionally, the provider emphasizes that a score of 40% does not mean that the AI has generated 40% of the text:

“Our AI detector provides a probability that a piece of content was AI or Original (human-generated). It returns a confidence score.

‍60% Original and 40% AI means the model thinks the content is Original (human-written) and is 60% confident in its prediction.” (Source: Originality.ai, “AI Detection Score Meaning”)

So, the tool rates the probability that some type of AI was employed during the generation of the text. That can also mean that the AI was merely used as a content planning tool or for spell-checking (Source: Originality.ai, “Most Common Reasons for False Positives With Originality”). In one post about false positives, Originality.ai even goes as far as to say, “When any amount of AI touches the content, it can cause the entire article to be flagged as AI.” (Source: Originality.ai, “AI Content Detector False Positives – Accused Of Using Chat GPT Or Other AI?”)

Why Is Detecting AI Use in Text Generation Important?

Many clients place great importance on getting content that’s not AI-generated, and with good reason. For one, text written by a real author is generally of higher quality and greater depth. Furthermore, the accuracy of content is also essential. That’s particularly the case with the so-called YMYL (Your Money Your Life) topics. With health-related, legal, or financial topics, the information has to be absolutely reliable and accurate. The subject matter expertise and the thorough research that a human author can provide exceed the capabilities of an AI by far.

There’s also a concern that AI-generated text will rank lower in search engines. For these reasons, Textbroker gives you a choice. We ALWAYS assume an order should be human created unless the briefings say otherwise. If you want to permit artificial intelligence during text generation, you can customize your briefing to say so.

This raises an important question: Does Google even care whether a text was written by a human or generated by an AI? Google itself states that content quality is the highest priority:

“Our focus on the quality of content, rather than how content is produced, is a useful guide that has helped us deliver reliable, high-quality results to users for years.” (Source:  Google Search’s guidance about AI-generated content)

The internet giant emphasizes that the focus is on user-oriented content. What matters is that your content provides added value to the users!

Google prioritizes quality content that provides value to readers.

What Is Google’s Stance on AI-Generated Content?

This brings up another question: Can Google reliably identify AI content? Worth mentioning in this context is Google’s E-E-A-T-Update, where the search engine operator has expanded their guidelines for quality evaluators. E-E-A-T is short for ”Experience,” ”Expertise,” ”Authoritativeness,” and ”Trustworthiness,” bringing us back to the added value mentioned above or what Google calls “Helpful Content.” Users should be able to rely on content being trustworthy, factually accurate, and unique as well as being provided by a person of experience and expertise. Google gives a higher ranking to content that fulfills these criteria. This concerns mainly text written by real authors or content that has, at least, been thoroughly and conscientiously edited by such authors.

Our Suggestions for Handling False Positives

It’s understandable that clients would like text generated entirely without AI assistance. And we would like to ensure this is the case. For this reason, we check EVERY text submitted through the Textbroker platform with our detector tool. Whenever our AI checker spots a supposedly suspicious text, we conduct an additional manual check.

In doing our checking, we rely on our experience, our competent editorial team, and the fact that we have become very familiar with our authors’ writing styles over the years. So, when it comes to correctly assessing texts, we rely not only on an AI detector tool but also on the combination of several rating criteria and the know-how of experienced employees.

Nevertheless, an external AI detector tool could suspect AI use when you check a text. As previously mentioned, suspected AI use does not automatically mean that the text was created by or with the assistance of artificial intelligence. If you explicitly ask for text within a certain AI score, utilizing a tool of your choice, you can make corresponding arrangements with an author via a DirectOrder. Of course, we are also available if you need assistance or if you should have further questions about the subject.

In Conclusion: Know the Limitations of AI Detector

If you’re a content publisher, the fact that such tools exist is good. After all, they provide assistance and offer a starting point. Yet, you should always take the results with a grain of salt and check for possible causes of false positives:

• Is the text even long enough for meaningful results?
• Have you selected the right tool for your language?
• Does the text require certain content, structures, and phrasing that could skew the results?
• Might the author merely have applied spelling or grammar checkers?

If you consider these aspects and question AI detector results with common sense, you’ll make a good assessment. And if you don‘t? In that case, we’ll be here for you — with our knowledge, experience, and trusted freelancer community.

What has been your experience with AI detectors? Share it with us in the comments section!

This post How (Un)Reliable Are AI Detectors? How False Positives Come About and How to Best Deal With Them first appeared on textbroker.com.