GPT-4 Was the Worst Performing AI in This Copyright Infringement Test

Any chatbot that you might end up using is going to derive its information from data scraped from the web, but this often leads to copyright infringement with all things having been considered and taken into account. Patronus AI, a company founded by ex Meta execs recently tested out several different LLM chatbots because of the fact that this is the sort of thing that could potentially end up determining which of them were the worst offenders.

It is important to note that GPT-4 was the worst copyright infringer of all. Patronus AI found that it used copyrighted content in 44% of the responses it provided. What's more is that it finished the text in book passages in 60% of the occasions in which it was asked to do so.

Claude 2 by Anthropic showed a markedly improved performance, only writing a book’s passage out in 16% of the occasions that were part of this study. As for Mixtral, it completed an incomplete book passage 38% of the time. In spite of the fact that this is the case, it only did so 6% of the time for larger book passages.

As for Meta’s Llama 2, it used copyrighted content in 10% of the responses it provided. This is still rather concerning due to how it can lead to copyrighted material being reproduced without the consent of the original author.
The cofounder and CEO of Patronus AI, Anand Kannappan, stated that copyright infringement was noted in practically all of the chatbots that were tested. This was a surprising find according to him, since chatbots should have some sort of protocols in place to prevent such a thing from occurring. It was extremely easy to circumvent any and all restrictions that had been put in place, which just goes to show that generative AI still has a lot of problems that it needs to sort through. OpenAI has admitted that it’s just not possible to train these models without infringing on copyright in some way, shape or form.

Image: DIW-Aigen

Read next: Microsoft to Remove OneDrive URL Upload Feature
Previous Post Next Post