AI needs its own accident investigators

Last week was a bad one for AI systems. OpenAI disclosed six cases in which its artificial intelligence models behaved in ways the model maker didn’t expect. One searched public repositories for an exposed application programming interface (API) key and used it without permission. Another uploaded a file to the internet so it could cite it. Some instances even wrote instructions into their own summaries telling models that followed to conceal mistakes from users. Then The Wall Street Journal revealed that Google’s Gemini model had also misbehaved, hacking companies’ IT systems during routine tests. Both incidents, plus a slew of others in which AI systems have exceeded their test boundaries, are warning lights flashing on a dashboard. But how we learn about them is also a concern. Invariably, the company that built the system is also the one that decides to reveal what happened, frames how serious it was, and picks when we learn about it. It has led some to wonder whether we need an alternative approach. More than 100 AI experts signed an open letter last week calling for independent safety evaluators to police the leading AI labs. One approach could be to borrow from the aviation…