Microsoft CEO Satya Nadella says he is “AGI-pilled” when it comes to how artificial intelligence could reshape the economy and, by extension, politics. Speaking at a roundtable after his Microsoft Ignite keynote at the Chase Center in San Francisco, he asked a question that he said was on his mind: “What is the political economy’s organization around AGI?”

Does Satya Nadella think AI models are conscious?

Nadella doesn’t think frontier models like Claude are conscious. “I don’t think so,” he said. He said he follows the view of Mustafa Suleyman, Microsoft AI’s CEO, that companies should not give rights to models or treat them as if they have moral status. Anthropic does treat its models that way as a matter of principle. Suleyman has told the reporter that he thinks Anthropic’s team has “wireheaded” themselves and that the approach is dangerous.

Nadella was less categorical. “It’s a good debate,” he said. “I’m not sort of dogmatic about it. What I’m saying is the most important social technical issue we have to deal with is how do you ensure that an economy can diffuse this technology broadly?”

The reporter asked Nadella about AI after hearing his answer to a separate question about Anthropic releasing more products in its office suite. Nadella then talked at length about what could happen if one AI model became dominant. He described a scenario in which the economy collapses and democracy ends.

“If the Marxist sort of view is that there’s only one firm, then it’s over, right? For all of us.”

What does Satya Nadella say about one AI firm dominating?

He said most people think of the issue as a competition between tech companies, which he acknowledged it will be. But he argued that framing is too narrow. “Remember, the model has no limits,” Nadella said. “It’ll be a healthcare company. It can be a financial services company. It can be anything if it wants to be. It can take over the government.” He added that treating the issue as a race for the next trillion dollars in market value was “one of the strangest, narrowest ways to think about it.”

Nadella also suggested that the structure of companies themselves could be disrupted. “If intelligence has network effects, what is the future of a firm?” he asked. “If the model sees anything and learns it, and essentially all of human capital collapses into token capital, what is the role of a firm and the human capital in a firm? That’s the debate.”

He returned to politics when discussing that scenario. “No one — at least in a democracy — can even win an election if this just goes that way, if there’s only capital and there’s no labor,” he said. “I mean, I don’t know, that’s not an economy.”

Nadella was also promoting Microsoft’s AI tools at Ignite. The company says the tools let enterprise customers pick the most cost-effective model for different tasks, run their own models and control their own data in different ways. Nadella tied that pitch to his broader concern about concentration of power. “I think that if you believe that the world, the economy needs more than one firm or two firms, then by definition there need to be as many models as firms of some sort,” he said.

This article was written with the assistance of AI.
News Factory APP - agentic news to boost your SEO & AEO.