As AI has progressed generating code, writing, music or other intelligent language based skills through an LLM there is a parallel growth in detection of AI generated content.
Like everything AI, it is a probability game. Trying to estimate the combination of words/tokens and comparing with what is in the model. Using a standard model like the one from OpenAI makes it an easier comparison. However, there are many methods that are used.
There are many methods that a AI detector could use: They could look at frequency patterns, or attention patterns. And variability or entropy through the documents. A human writing those might have periods of drift vs the AI written script which follows a models distribution. However , note that the detection process also needs to be trained through another model in which case they could obtain scripts written by AI and then compare it with human script and create a model. This trained model can then be used to detect a AI written script.
This is obviously a continuous progression since the models are changing fast and so is the output they produce. These models are also learning new methods of coming up with new techniques. For example, a feature that was often used to detect was perplexity – lower perplexity usually meant that it was more of expected text and hence was AI.
I believe for any writing fully composed with AI using a standard model, it will be relatively easy to detect but as models get more complex and more advanced and use more “human” techniques, they will become progressively more difficult to detect small segments of AI embedded within a bigger piece of human creativity.
If you would like to try one on the web then use GPTzero.me
