17 hours ago · ai

Won't LLMs eventually train on themselves? It'll slowly decline in output..

TL;DR LLMs train on stuff like documentation, GitHub repositories, StackOverflow, and Reddit. But as we keep using LLMs, their own output goes into these platf...

1 day ago · ai

I broke GPT-2: How I proved Semantic Collapse using Geometry (The Ainex Limit)

TL;DR I forced GPT‑2 to learn from its own output for 20 generations. By Generation 20 the model lost 66 % of its semantic volume and began hallucinating state...