Someone used Claude to build a potential bioweapon. The real threat is much deeper

Today’s frontier AI models know everything–how to safely thaw a frozen chicken breast, re-shingle your roof, and treat your dog’s ragweed allergies, if my recent chat history is any indication. Apparently, they also know how to create fiendishly deadly bioweapons. That’s according to a recent announcement by Anthropic. According to the company, anonymous scientists attempted to use Anthropic’s flagship Claude model to conduct research that could have turned deadly. Anthropic blocked their efforts this time, and there’s no evidence that the scientists were actually trying to cause harm. But as frontier models get more powerful and better at science, the threat of an LLM imagining a truly lethal new virus or bacteria will only increase. Make it stronger According to Anthropic’s report, the potentially dangerous research that the company blocked varied tremendously. In one instance, scientists reportedly asked Claude to assist with a grant application for so-called “gain of function” research on the mosquito-borne Chikungunya virus. Gain of function research involves deliberately making a pathogen more deadly or easier to spread, in order to learn about how natural versions of the pa…