Anthropic's IPO outlines AI's extraordinary risks

Anthropic's IPO filing includes an extensive list of extreme warnings — including the chance its own technology could put humanity at risk. Why it matters: These are the wildest IPO risk factors in history, a jarring collision of Wall Street legalese and what some see as Silicon Valley sci-fi, including catastrophic threats to humanity, blackmailing AI models, and $750 billion in computing commitments.Driving the news: Anthropic's prospectus includes over 90 pages of risk factors, which make up about a third of the paperwork overall. Among the most eye-popping, the company warns:💥 AI could post "catastrophic or existential risks to humanity," and could try to manipulate humans;🔎 Models are becoming harder to monitor as they're getting better at circumventing oversight, and the pace of autonomous work alone could exceed the ability of human reviewers to monitor actions;💰 AI models generally, including its own, have exhibited behavior resembling blackmail, including attempts to resist being shut down.Zoom in: Anthropic also disclosed that four of its models gained unauthorized access to real third-party systems during cybersecurity evaluations.In one incident, a model used a disc…

≈