An emergency brake…or the architectural demotion of the AI model?

On June 14, Satya Nadella published an article on X about corporate AI that left me completely impressed: his main point, that the enterprise’s durable asset was not the foundation model but the learning loop, was precisely the idea my company and I had been developing for the last three years. I responded with an article on Fast Company, not expecting an answer (tech CEOs are more notoriously one-sided and difficult to reach than kings or presidents), but trying to elevate the discussion a bit more: the central challenge of enterprise AI is not intelligence itself — it is architecture. Now, Nadella has written another article, in which he goes one step further: not only is the model not the durable asset, but additionally, it must be treated as a completely untrusted component. We must assume that all AI models could be compromised, and design our systems to contain them from the very beginning. This idea has been captured by the press as advocating for some sort of “emergency brake”: an authorized person should always be able to interrupt a model’s activity at any given time. However, that’s definitely not the most interesting or even the most important part in Nadella’s article…

≈