In recent years, consumers and the media have focused a lot of attention on how physical tech products are manufactured, who makes them and where the materials come from. Now that generative AI has taken the world by storm, we have to ask the companies behind the leading LLMs (Large Language Models) a similar question: where does your training data come from and is it ethically sourced? The answer is likely “no.” Using generative AI today is like buying from a seedy pawn shop. The goods, aka training data, could be legit sales from the owner, high-quality merchandise that was stolen from a boutique or low-quality shlock that was pilfered from a warehouse full of knockoffs. The popular models are built on a foundation of stolen intellectual property: copyrighted texts, images and videos taken without permission or compensation from sources both good and bad, and used to “train” the software so that it can create similar content. In just the past two weeks:
- Popular image-generator Midjourney released version v6 of its software, which can produce images from popular movies that are nearly identical to specific frames from the actual films.
- CSAM (child abuse images) were found by Stamford researchers in the popular LAION image dataset that many LLMs use for training.
- The NY Times filed a sprawling copyright infringement lawsuit against OpenAI and Microsoft, alleging that ChatGPT and Microsoft’s derivative products have “a business model based on mass copyright infringement.”
Let’s start with the most recent of these developments: the NY Times lawsuit. In the 69-page filing, the news organization explains how OpenAI built its LLM models using Common Crawl, a free dataset that it claims contains “at least 16 million unique records of content from the Times.” It also cites a bevy of instances where GPT-4 and its Browse with Bing feature were able to repeat content taken verbatim from NY Times content.
Full commentary : AI’s Dreadful December: Lawsuits, plagiarism and child abuse images show the perils of training on data taken without consent.
