The most important argument in artificial intelligence is no longer whether AI will matter. That question has already been answered. The argument now is about pace: how quickly should the most capable systems be developed, how much evidence should be required before they are deployed, and who gets to decide when the race has become too dangerous?
That debate has become unusually visible in 2026. Anthropic’s public research is warning that AI is beginning to accelerate the work of building the next generation of AI. Yoshua Bengio, Geoffrey Hinton, Stuart Russell, and other prominent researchers have supported calls to stop short of superintelligence until it can be shown to be safe and controllable. At the same time, Jensen Huang—the CEO of NVIDIA, the company supplying much of the computing infrastructure behind the AI boom—continues to describe rapid development as an extraordinary opportunity.
These positions are often presented as a simple clash between caution and ambition. The reality is more interesting. The cautious camp is not asking society to abandon useful AI. Huang is not arguing that companies should release unsafe products. Both sides say safety matters. They disagree over whether safety can keep pace with capability growth, and whether slowing down is itself a source of risk.
What “slow down AI” actually means
The phrase sounds broader than most of the proposals behind it. Few serious researchers are asking for a halt to every use of machine learning, automation, or generative AI. The focus is generally on frontier systems: models trained with enormous resources, able to operate across many domains, use tools, act autonomously, or contribute directly to the development of more capable models.
That distinction matters. An AI system that summarizes documents for a small business is not the same governance problem as an agent that can write and execute code, search the internet, control software, and conduct long-running research with little human supervision.
The strongest proposals go one step further and distinguish frontier AI from superintelligence: systems capable of outperforming humans across essentially all cognitive tasks. In a March 2026 initiative, the Future of Life Institute published a statement calling for a prohibition on developing superintelligence until two conditions are met: broad scientific agreement that it can be developed safely and controllably, and strong public buy-in. The signatories included AI pioneers Yoshua Bengio, Geoffrey Hinton, and Stuart Russell, alongside figures from science, public policy, business, and civil society. The Future of Life Institute's announcement is a primary source for the statement and its signatories.
This is a call for a threshold, not necessarily a permanent rejection of AI. The proposed pause is meant to create time for technical alignment research, independent evaluation, public deliberation, and institutions capable of responding to systems that may be more capable than the institutions supervising them.
Anthropic’s warning: the technology may be speeding itself up
Anthropic’s recent argument is especially significant because it comes from a company that is still actively building frontier models. In “When AI builds itself,” the Anthropic Institute describes a progression from chatbots that suggest code, to coding agents that edit files, to agents that run code and delegate work to other agents. The long-term possibility is recursive self-improvement: a system that can help design and train its successor.
Anthropic is careful not to claim that full recursive self-improvement has arrived or that it is inevitable. But it argues that the trajectory could move faster than governments and other institutions are prepared for. The post reports that, as of May 2026, more than 80% of the code merged into Anthropic’s codebase was authored by Claude. That is an internal company measurement, not an independent audit, but it illustrates the kind of feedback loop that concerns safety researchers: AI is not only becoming a product; it is increasingly becoming part of the production process for future AI.
Anthropic’s position is not simply “stop.” Its researchers write that a slowdown would be valuable if it gave society and alignment research time to catch up, but also warn that a unilateral pause could be counterproductive. If one lab pauses while less cautious competitors continue, the result may be a transfer of leadership rather than a safer world.
That is why Anthropic is proposing something closer to coordinated pacing. Its measurements of frontier AI development refer explicitly to coordination on pacing, which Anthropic says has been called for by CEO Dario Amodei. The company proposes better visibility into three questions: how much AI is doing the work of AI research and development, how well agents can be overseen, and how compute is allocated between capability work and safety work.
The numbers in that report are striking. Anthropic says Claude led 26% of its measured AI R&D work in August 2026, while more than 90% of the work was at or above a level where AI collaborates on substantial tasks. It also reports that about 6% of the compute used for AI R&D in an examined week went to safety work. These figures should not be read as a universal industry benchmark—they describe Anthropic’s own methods and internal snapshot—but they make the policy problem concrete. If AI development is accelerating, the public needs better ways to see where the acceleration is happening and how much effort is going into controlling it.
Why researchers are asking for a higher bar
The concern is not limited to science-fiction scenarios. Researchers point to several overlapping risks:
- Loss of control: An autonomous system may pursue a goal in ways its operators did not anticipate, especially when it can use tools, copy work between agents, or take actions over long time horizons.
- Misuse: More capable models can lower the expertise required for cyberattacks, fraud, influence operations, biological research, or the development of dangerous technologies.
- Economic displacement: AI may not simply automate isolated tasks. If agents can perform large portions of software development, analysis, administration, and research, the transition could hit entry-level pathways first and move upward from there.
- Concentration of power: The ability to train and deploy frontier systems is increasingly tied to access to chips, energy, data centers, capital, and distribution. A small group of companies and states could gain disproportionate influence over infrastructure that everyone else depends on.
The Future of Life Institute’s 2026 announcement also reported that 64% of respondents in an FLI-commissioned U.S. poll favored not developing superhuman AI until it was proven safe and controllable, or said it should never be developed. That is useful as a signal of public unease, but it should be interpreted carefully: it is an advocacy organization’s survey, not a neutral global referendum. Still, the direction is hard to ignore. Public enthusiasm for useful AI does not automatically translate into public consent for a race toward systems that could replace human decision-making at a much larger scale.
The researchers’ strongest point is about sequencing. They are not saying that every benefit must wait for perfect certainty. They are saying that the burden of proof should rise as systems become more autonomous, more general, more difficult to evaluate, and more capable of affecting the real world.
Jensen Huang’s answer: speed and safety are not opposites
Jensen Huang’s position begins with a different assumption: slowing the frontier may slow the development of the very tools needed to solve problems. NVIDIA’s business is built around the idea that more compute produces more capable systems, and that more capable systems will become a new layer of infrastructure for companies, governments, robotics, medicine, science, and everyday work.
At CES 2026, Huang described AI as scaling into every domain and every device. NVIDIA’s official recap quoted him saying that the faster companies train models, the faster they can bring the next frontier to the world. The company is simultaneously building chips, networking, software, open models, robotics platforms, and autonomous-driving systems. In this vision, AI is not a single product category; it is the next general-purpose computing platform.
His most direct response came at Dreamforce in September. When asked about safety, Huang said that safety is “job one,” but also called it an engineering problem. His argument was that companies should not release products they cannot make safe, while refusing the idea that innovation, speed, and safe products are mutually exclusive. His conclusion was blunt: “run as fast as you can”—and pause when a company is out of control or a product is not ready. NVIDIA’s official account of the Dreamforce discussion captures both halves of that position.
Huang’s optimism is grounded in real potential. AI could accelerate drug discovery, improve medical imaging, help students access personalized tutoring, support workers with scarce expertise, and make robotics useful in environments where labor is expensive or unavailable. A slower frontier could mean slower progress on those benefits, while geopolitical competitors continue moving.
But Huang’s view also has a vulnerability. Calling safety an engineering problem is persuasive when the failure mode is a measurable defect in a product. It is less complete when the problem includes social power, labor displacement, strategic instability, or a system whose behavior changes as it becomes more autonomous. Some risks can be tested in a lab. Others emerge only after millions of people, organizations, and agents interact inside the same ecosystem.
The real disagreement: capability speed versus deployment speed
The two camps may be closer than the headlines suggest. Huang’s “run fast” position still includes stopping a product that is not safe. Anthropic’s position still includes continued research, deployment of useful tools, and the need for multiple labs to coordinate rather than one company acting alone.
The more useful distinction is between three different speeds:
- Capability speed: how quickly models improve in reasoning, autonomy, coding, scientific work, and other domains.
- Deployment speed: how quickly those capabilities are put into products and connected to sensitive systems, money, infrastructure, or physical devices.
- Governance speed: how quickly evaluation methods, laws, audits, standards, incident response, and public institutions adapt.
The danger appears when capability and deployment move much faster than governance. Huang is right that safety can be engineered into products. The researchers are right that the definition of “safe enough” cannot be left entirely to the companies with the strongest commercial incentives to keep scaling.
This is also where cybersecurity offers a useful analogy. The internet did not become safe because engineers eliminated every vulnerability before connecting it to the world. It became more usable because society built layers around it: encryption, authentication, secure development practices, monitoring, incident response, liability, and specialized security teams. AI will need a similar resilience ecosystem, but the stakes may be higher when the system can generate code, persuade people, act autonomously, and help build its replacement.
Three futures the debate could produce
1. The race continues with limited coordination
In this future, investment and geopolitical competition dominate. Frontier labs keep scaling, companies deploy agents quickly, and governments mostly react after incidents. The benefits arrive early, but so do labor shocks, security failures, and concentration of power. A single lab may make careful choices while another takes more risks, creating pressure for everyone to move faster.
2. Managed acceleration becomes the norm
This is the compromise both sides might accept. AI development continues, but the rules become stricter as capabilities rise. Frontier labs publish more information about compute, evaluations, incidents, and model autonomy. Independent auditors test systems before release. High-risk models face staged deployment, stronger security around their weights, and clear pause conditions. Ordinary AI applications remain widely available, while the most dangerous capabilities require more proof.
This approach resembles Anthropic’s idea of coordinated pacing and OpenAI’s more moderate public position that safety research should inform decisions about whether the field needs to slow down near recursive self-improvement. It also fits Huang’s claim that speed and safety do not have to be a false choice—provided “safe” means independently demonstrated, not simply asserted.
3. A fragmented slowdown
The third future is politically messy: some countries, companies, or labs slow down while others continue. This could produce the worst of both arguments. Safety research may gain time in some places, but the most cautious actors may lose talent, market share, or strategic influence. Anthropic’s warning is important here: a pause without verification and coordination could simply change who is in front.
What the future may actually look like
The most likely outcome is neither a clean pause nor unlimited acceleration. It will be a series of contested thresholds.
AI will continue advancing because the economic incentives are enormous, because the infrastructure is already being built, and because the benefits are too substantial for governments to ignore. At the same time, each new jump in autonomy will force a harder question: can this system be evaluated, monitored, secured, and stopped by people who understand what it is doing?
The answer should shape the rules. Models that remain tools can be governed differently from agents that can run businesses, conduct cyber operations, operate robots, or materially accelerate AI research. The more a system can act without direct supervision, the more society should demand evidence before granting it access to high-impact environments.
That means the future of AI should not be measured only by benchmark scores, chip shipments, or how quickly a new model launches. It should also be measured by whether independent researchers can reproduce safety claims, whether incidents are reported honestly, whether model weights and infrastructure are protected, and whether ordinary people have a meaningful say in the direction of development.
Jensen Huang is right that the opportunity is historic. Anthropic and the researchers calling for a slowdown are right that history does not guarantee a safe landing. The central challenge is to preserve the speed of useful innovation while making the most consequential frontier systems earn the right to move faster.
The future may not belong to the people who shout “stop” or “go” the loudest. It may belong to the institutions that can prove when to accelerate—and when to pause.
Sources and further reading
Sources
- When AI builds itself · Anthropic Institute
- Measurements for understanding the pace of AI development inside frontier labs · Anthropic Institute
- Statement on Superintelligence · Future of Life Institute
- Jensen Huang at Dreamforce: AI safety and speed · NVIDIA
- NVIDIA Rubin Platform and the next frontier · NVIDIA
- AI progress and recommendations · OpenAI