In an unsealed court filing in the New York Times v. OpenAI copyright lawsuit, Microsoft and OpenAI executives admitted that large language models were trained on stolen content and have created a 'doom loop' destroying the web and the livelihoods of content creators. Internal documents describe the practice as 'an astonishing theft of unprecedented proportions' and 'the largest theft of labor in human history,' with AI products cannibalizing traffic from the sources they were trained on.
The article argues that while Jev appears to be a classifier model, dismissing it as 'just a classifier' overlooks its impressive generalization across diverse tasks like email classification, gaming, and trading. The author suggests Jev's breakthrough likely stems from superior training data and design rather than novel algorithmic innovations, comparing it to successful prior releases like Stable Diffusion and ChatGPT.
Unsealed court documents from the New York Times' lawsuit against OpenAI and Microsoft reveal internal communications showing the companies were aware they were creating a 'doom loop' that would damage the web, characterized their data scraping as 'the largest theft of labor in human history,' and acknowledged their models memorize and regurgitate copyrighted content verbatim despite knowing this violates fair use principles.
The article argues that unaddressed concerns from artists about AI training data have festered into widespread anti-AI sentiment that pollutes public discourse. The author warns that mathematicians now have similar grievances about AI, and if their modest demands aren't met promptly, their influential social position could amplify backlash similar to what occurred with artists.
Gig workers around the world, including Mohamad Dunggio in Indonesia, are earning money by recording themselves performing household tasks from a first-person perspective to train robots. These egocentric videos help robotics researchers teach humanoid robots how to handle domestic tasks, with companies like Appen collecting tens of thousands of hours monthly from workers across multiple countries.
In November 2024, the author fine-tuned a small language model on r/SkincareAddiction comments and built a classifier to filter training data by usefulness, objectivity, and quality. The article explains different approaches to text classification—regex, vectorization, LLM prompting, and teacher-student distillation—with the author using the last method to label ~120,000 comments for training.