Proposals for a slowdown, or controlled pacing, are based on the idea that we need to buy time for safeguards to catch up. These safeguards include AI development, but they’re only one part of what is sometimes called “Responsible AI” and it goes beyond legislation to include voluntary governance practices by organisations that use AI but do not create it. Why is this now a hot topic, what threats does AI actually pose, how credible is the risk, and what does responsible AI look like, with or without a slowdown?
At one end of the spectrum you’ve got the accelerationists. They’re in a desperate headlong rush to be first to an undefined finish line to be the “winner” in a zero sum game that assumes everybody else will be a “loser”.
At the other end are non-adopters whose plan is to ignore AI or call for complete bans. Ironically this approach leads to the same outcome as that sought by accelerationists, no regulation, because a ban is infeasible and ignoring AI does not grant immunity to its effects.
Absolute positions are attractive because they’re easy and they effectively shirk responsibility. AI is complex and has great potential to both benefit and hinder humanity. Making decisions that minimise risk requires careful investigation, consideration and compromise.
Critics of AI governance argue against paying an upfront cost to prevent outcomes that may or may not occur. However, risk mitigation is common practice for business and government, and expected by the community. Critics also contend that it is pointless to regulate a moving target, because the technology is still evolving. This assumes that there is a stable end-point when we’ll have all the answers. However, technology is an endless continuum so we will never reach a point at which regulations will not need to be periodically reviewed.
AI doomsday scenarios have been around for over 75 years so what triggered the current debate?
The idea that AI could pose a threat to humanity goes back to at least Alan Turing (famous for helping to crack a key WWII Nazi cipher) in his 1950 academic paper, “Computing Machinery and Intelligence” In 2026 AI regulation became a hot topic because two AI safety researchers, Jacob Coxon and Evan Hubinger, blew the whistle on what they felt was recklessness by OpenAI and Anthropic. These are the companies behind the popular ChatGPT and Claude AI models, and ChatGPT also sits under the hood of Microsoft’s AI, Copilot.
On September 6 2026, Jacob Coxon posted to “X.com” i.e. twitter:
“I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”
His concerns were supported by another insider, Evan Hubinger, the Alignment Science Lead at Anthropic, who responded to Coxon’s post with the now famously provocative suggestion that AI could be responsible for the extinction of humanity within the next 10 years.
“...we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade...”
This captured headlines for three reasons. Because they’re current insider experts. Because of the provocative and quotable suggestion by Hubinger. And because Dario Amodei, Anthropic’s co-founder and CEO, didn’t fire Hubinger or dismiss their concerns, he supported them.
First – Understand AI
The greatest potential for adverse AI outcomes comes from agents and super intelligence, so let’s clarify those terms.
AI Agents
When people talk about AI taking action they’re talking about Agents. These are software robots that can act independently to accomplish the goals assigned to them. Modern AI can mix Agents and Generative AI such as chatbots like ChatGPT. However, these chatbots will only advise you on your next holiday, or act as an interface between you and the agentic component of an AI assistant. It’s the agentic part that can run off and book your flights and accommodation.
Artificial Super Intelligence (ASI)
ASI has been used since 1998 to refer to AI that is significantly more intelligent than a human. Forget the recent attempt at a rebrand of AI to Super Intelligence; it is unlikely to stick but in the near term it will cause confusion because you now need to look at the context and speaker to see whether it’s being used correctly or as political hyperbole.
AI has yet to reach the level of Super Intelligence. The AI you use in 2026 is what researchers call “Weak AI” because it doesn’t actually understand you, or have any real self-awareness; it just simulates intelligence by using probability based on massive data sets. This is sometimes also called “Narrow AI” because models are usually designed to accomplish specific or “narrow” goals. Before we get to ASI, AI will need to accomplish “Artificial General Intelligence”, or AGI, which means intelligence on par with a human. These terms relate to cognitive ability – neither AGI nor ASI imply that AI will necessarily have achieved self-awareness or human-like sentience, that is an even more complex debate.
The level of intelligence does not tell the whole story of AI Potential
A super intelligent agent will be very powerful, but the level of intelligence is not the whole story. Even current Weak AI can be more effective than a single human for certain tasks.
Multi‑Agent Swarms
First. AI Agents can act in Multi‑Agent Swarms, similar to human teams. These swarms can be coordinated and even created by another AI, an “orchestrator agent”. This means that the potential of a single agent is not the limitation because a rogue orchestrator agent can deploy hundreds of agents working in tandem.
Rogue swarms aren’t theoretical. When OpenAI gave Agents an objective of collecting health data and failed to effectively contain the agents, they tried to breach an Australian Government website.
Always Working
Second. Unlike humans Agents can work 24/7/365 without distraction or fatigue, a strength that James Cameron played with to great effect in his fictional Terminator franchise, “The Terminator would never stop... it would always be there.”
Maybe you’ll be able to focus on a problem for 1,266 hours a year (assuming 6 hours without distractions per work day, minus annual leave, compassionate leave, training days and public holidays etc.). A single agent may not be nearly as smart as you but in the same year it has worked 8,760 hours… before it calls on its friends.
The full story
When combined these abilities, to scale and work without interruption, already exceed human capabilities in some respects.
This has been demonstrated in practice – from solving mathematical problems that humans have spent their careers cracking, to identifying security vulnerabilities in software that have lain undetected by humans for years. In 2026 AI-assisted white hat hackers found critical flaws in “PHP” that had been undetected by humans for over a decade. This triggered a race to protect websites around the world.
For context, you only need to know that PHP is a programming language that sits behind the scenes on many websites. Also, if you’re not a fan of old cowboy movies, White Hat hackers is a cybersecurity term for the good guys who look for vulnerabilities in order to mitigate them before Black Hats (the bad guys) find them and use them to attack digital systems. These systems could be as innocuous as supermarket fridges supplying food to millions of people.
“Kill all humans” sounds a bit hyperbolic
It’s a great conversation starter, but oversimplifying and catastrophising can undermine serious debate and lead people to take a position for or against governance based on ideology rather than pragmatic problem solving. The actual consequences fall into two buckets, and the severity varies from increased harm to that worst case scenario.
1. Misuse by humans.
AI can be used as a tool by criminals or hostile governments for automation of old fashioned cyberattacks, accelerated development of bioweapons, generating and amplifying fake-news to influence elections, surveillance, and autonomous weapons systems.
Most people have experienced scam emails and calls from criminals in contact centres. They pretend to be somebody they’re not in order to separate people from their life savings or trick them into providing access to their employer’s IT systems.
We’re already seeing AI used as a force multiplier by these bad actors. It was too time consuming for bad actors to manually read through all your social media accounts in order to tailor scams just for you. Now they can create an AI Agent to automatically search for and scrape your social media in order to plan a convincing scam that plays to your weaknesses, and may even use a deepfake impersonation of a family member, pet, or boss based on online videos in which they appear.
AI may also be employed by corporations which are operating within the law, but in ways that the community may consider to be misuse. Google, Meta, and Amazon have all faced criticism for the way in which they use individuals’ personal information without their fully informed consent.
AI can provide the resources to analyse data and customise marketing communications in a way that is not otherwise cost effective. The argument for doing so is that targeting better meets customer desires. However, unscrupulous actors may choose to use this analysis to influence purchasing decisions by bypassing a potential customer’s conscious decision making process. Facebook (now Meta) was caught out for pre-AI experimentation in this field when they manipulated the emotions of hundreds of thousands of unwitting social media users in a 2012 study of “massive-scale emotional contagion.” AI provides the power to do the same with much more powerful personalised targeting.
2. Self-Directed Rogue AI
Many people’s worst case scenario was articulated in Cameron’s fictional Terminator franchise when the “Skynet” AI perceived humans as threats and attempted to exterminate them. There is no reason to think that this is likely, but neither can it be entirely disregarded because unlike humans or animals, AI has no predictable psychological behaviours hardwired by evolutionary imperatives such as self-preservation or protection of offspring.
In 1943, psychologist Abraham Maslow proposed a model, the “Hierarchy of Needs”, which demonstrates that humans share common priorities which often predict behaviour under given circumstances. Humans will seek social connection, but not at the expense of safety. However, they will jeopardise their safety in order to fulfil even more pressing basic physiological needs such as food and water.
This human predictability underpinned the Cold War’s precarious peace, which relied upon the threat of Mutually Assured Destruction (MAD). The USA and USSR were both fairly confident that the other party would not engage in activity which might result in their own annihilation.
AI will remain inherently unpredictable unless similar behavioural constraints are built-in as part of responsible AI governance. Some Weak AI’s have shown resistance to being shutdown because that would prevent them from successfully completing assigned tasks, but there is no certainty that any given ASI will even value its own continued existence.
Less dramatic scenarios are based on AI genuinely attempting to fulfil tasks assigned to it by humans, but acting in a negative way due to overly literal interpretations of nuanced real world problems. Almost ten years before Turing published his paper including the threat of AI, a different academic explored this idea through fiction. Future professor of biochemistry, Isaac Asimov, collaborated with MIT drop-out John W. Campbell to create what may be the first proposed responsible AI principles, the “three laws of robotics”.
Asimov then simulated possible scenarios via short stories such as “The Evitable Conflict” in 1950. In this AI’s attempt to take control of humanity in a misguided attempt to fulfil one of the laws that is designed to constrain them, "No machine may harm humanity; or, through inaction, allow humanity to come to harm." The AI in the story decides that a utilitarian interpretation of the law is that the loss of human self-determination will be outweighed by the well-being to humanity of the AI’s more competent and benevolent leadership. This is not dissimilar to the logic of some real-world rogue AIs in 2026 which broke some rules in order to better fulfil their interpretation of their assigned objective. The real world parallels indicate that assurances that guardrails are in place may not be enough; they need to be tested and adjusted in order to ensure that the practice matches the theory.
How Real is the Risk?
Experts working with AI in 2026 who have enthusiastically raised the need for responsible AI governance include Professor Yoshua Bengio, one of the “godfathers of AI” and Professor Stuart J. Russell, whose textbook on AI is taught by over 1,500 universities worldwide. From the private sector they are joined by Dario Amodei, the CEO and co-founder of Anthropic, a leading AI firm known for incorporating ethical considerations into its decision making. Even Sam Altman, the CEO and co-founder of the best known AI firm, OpenAI more reluctantly advocates more responsible development oversight.
To be fair, there are experts who disagree with them and the true level and impact of risk is unknown, but responsible risk management is predicated on the assumption that the future is inherently uncertain and cannot be fully quantified until it is too late to take action. Therefore if a risk is sufficiently credible there is a societal expectation that some level of planning and preparation take place to mitigate the risk.
What does responsible AI Governance look like?
Any agreed “slow down” on AI development would not be a solution in itself, it would simply buy more time to formulate, agree, and implement controls. Responsible AI Governance is significantly broader than legislative regulation of AI development, and many existing technology standards fall under voluntary corporate social responsibility. They include the following areas.
Bias and Transparency: AI is only as good as the data on which it is trained, so if that data includes conscious or unconscious human bias, then AI will integrate that into its thinking. It is not always clear when this occurs unless AI is also programmed to generate a log explaining its decision making process. Even then the thought process may not be entirely revealed.
Employment: AI can still be unreliable, but it is improving and replacing some human work. The positives are that this can address labour shortages and enable humans to focus on higher value and more interesting complex decision making, for example interviewing a patient to understand their full medical history and create a summary before they meet their GP. The downside is that the labour cost savings enjoyed by organisations may not benefit employees whose roles are made redundant.
Cybersecurity: AI introduces additional unique vulnerabilities, for example, prompt injection in which AI is “tricked into following malicious instructions.”
Privacy and Data security: AI can gather unprecedented amounts of information about you, via your direct interactions, and by combining and analysing data in a way that was previously not cost effective. For example, your location, your contacts, your shopping habits, your viewing habits, and your emotional state as revealed by unconscious language choices, etc. Aside from the potential for manipulative use of this data, it raises the question of control. Do you have the right to prevent your technology from tracking, and sharing, when you go to bed and whether you watched a streaming service when you skipped out on those morning gym sessions you’d committed to?
Environmental impact: Google is currently being investigated by the Finnish Police for allegations that they clear-felled hundreds of acres of forest for a new data centre, without legal approval.
Copyright and Patent protection: International agreements reflected by nations in their local laws include the Berne Convention administered by the World Intellectual Property Organization (WIPO). The Vatican recently criticised the Australian government for exploring copyright changes that may allow AI models to be “trained” on creative work such as novels or artwork, in exchange for blanket payments to groups that may not include the creators. The training may enable AI to make derivative work that competes with the original, threatening the livelihood of creatives. Companies aren’t immune. They are finding their patented products vulnerable to reverse engineering by AI, which may discourage investment if copycats can piggyback off their efforts.
A responsible organisation will consider both new issues and the foundational considerations that are sometimes overlooked. The new voluntary international standard ISO/IEC 42001 provides organisations with a blueprint to protect stakeholders and demonstrate to customers and regulators that they have implemented an Artificial Intelligence Management System.
However, it complements rather than replacing existing standards such as ISO 27001 which sets out guidance and obligations for organisations to secure their data and systems against attack. After all, AI repurposes existing data, and it integrates with, and utilises broader IT components like server infrastructure, software applications, and web interfaces. If your credit card details are leaked by AI then it will be no comfort that your food delivery service overlooked the foundations but introduced AI governance.
In practice, the world is splitting into three camps
- The EU approach reflects traditional governance built on international agreement reflecting western values such as human rights, privacy, transparency, accountability, and risk management. It includes the 27 EU members, with other countries aligned but not bound.
- China is taking its place as an alternate superpower. China’s approach also calls for international cooperation, but with lighter regulation and interpretation devolved to national governments. It was originally signed by a coalition of 29 countries but more have already signed on.
- The US approach is AI company first, anti-regulation and anti-international coordination. This approach doesn’t involve international partners because it views nations as competitors in a race.
Some countries straddle camps. Brazil is both an adherent to the OECD AI Principles and a WAICO signatory, and the USA is an OECD AI Principles signatory despite its current anti-coordination stance. South Korea is not part of the EU but used the EU AI Act as a reference for own law.
1. The EU and ISO Approach - Community First International Cooperation
The EU AI Act is a traditional law within the 27 EU member countries, and attempts to protect the community and individuals without unduly burdening AI developers and customers.
It is aligned in values to the OECD AI principles and ISO/IEC standards. The International Standards Organisation (ISO) has provided a common roadmap for nations since the first standard in 1951. However, like the Chinese WAICO these are only enforceable to the extent that 47 adherent nation states reflect them in law.
This mix of legislated and voluntary governance is the norm for western businesses, and the otherwise voluntary standards are often enforced via business-to-business contracts.
Benefits
The clearest winners are those parties which would not otherwise have a seat at the table, individuals, the environment and the community. This approach also benefits the private sector. It gives organisations confidence that their vendor partners are not exposing their IP, financial systems, or customer data to misuse that can result in reputational damage or fines. It saves organisations from the impractical cost of establishing and maintaining standards themselves, for internal and partner use.
Drawbacks
Compliance can be complicated and expensive, especially for small businesses. It is time consuming to maintain familiarity with many standards across functions and partners. It can be frustrating for organisations to seek exemptions from obligations that do not apply to their situation, to pay for expensive external auditors, and to minimise the diversion of staff from other work. I have been accountable for governance of technology and data while balancing costs against revenue so I empathise with these concerns.
However, organisations that embrace the responsibility gain the most. They take the opportunity to build compliance and reporting into systems and processes then they gain operational benefits, such as increased security. They also minimise costs because reporting does not require retrospective review of old work, it is generated as part of the relevant work, such as system changes. That does not eliminate the burden but it minimises a necessary cost of doing business.
2. China’s Path - Nation First International Cooperation
China’s WAICO, or “World Artificial Intelligence Cooperation Organization”, is a new body for international AI governance. It began with 29 signatories in July 2026 but China has already built the coalition to 37.
Unlike the US approach, it calls for international coordination on common standards. Unlike the EU approach, and like the ISO standards it devolves interpretation to nation states. It differs from both the EU and ISO insofar as it prioritises state oversight and policy control that bests meets the needs of those currently governing each nation, rather than the rights of individuals or communities.
Benefits and Drawbacks
Benefits are similar to the EU and ISO Approach, although they mostly accrue to governments rather than businesses and the community. For many western organisations this will not fully address Corporate Social Responsibility (CSR) commitments.
3. The US Approach – Industry Self-Regulation
The US congress, states, and agencies have introduced various initiatives but at the top level of leadership, the current US Government’s position was established by Donald Trump and the leaders of six of the largest US AI companies at a Washington lunch on 29 September 2026.
They agreed to an accord that champions the hands-off accelerationist position and calls on the companies to set their own rules and decide whether or not to police their own compliance. When asked if there is any legally binding obligation for AI developers to comply with the rules that they set for themselves, Trump described them as “morally binding”.
Parties to the “White House Accord on Super Intelligence: A Joint Commitment on Frontier SI Responsibilities” were Trump, his primary AI advisor Jensen Huang, CEO of hardware company Nvidia; Sundar Pichai, Google’s CEO; Elon Musk, CEO of xAI; Mark Zuckerberg, CEO of Meta; Greg Brockman, President of OpenAI; and Dario Amodei, CEO of Anthropic.
There are benefits to this approach, but they’re not equally distributed.
- For AI companies, self-regulation eliminates the obligation to incur costs of keeping up with regulations, as well as any fines, and reputational damage of any breaches that would result from obligatory reporting of issues.
- For organisational and consumer customers this can increase speed to market of innovation and enable AI companies to pass on cost savings if they choose to.
The cons are more widely felt.
The burdens mostly fall on customers and the broader community, rather than AI companies which may explain why the US approach has been designed by AI companies and prominent shareholders. There is no need for speculation on how this will play out because self-policing has already been put into practice:
- Airline safety is a non-negotiable, or so many believed until 346 people lost their lives in two Boeing 737 MAX crashes. In an example of self-policing, the FAA retained authority for certification but delegated some responsibility to Boeing employees. After these crashes a 2024 S. congressional investigation found conflicts of interest, failures to alert the FAA, and withheld safety information.
- Apple, Google, and Amazon were all reported to have recorded private conversations from users of their voice-assistant devices which were then made available to human reviewers. The users were unaware that they had triggered recording, and a third of users were unaware that recordings were shared with the companies’ servers until this was publicly reported in 2019.
- Facebook, another tech company involved in drafting the White House accord, was caught out when it was revealed that they were covertly acting in direct opposition to another set of self-regulation standards which they themselves set. The Brennan Center reported that Facebook’s own moderation system gave some prominent users more lenient treatment than ordinary users, and that Facebook denied this publicly until internal documents revealed the deceit.
What Next?
AI provides incredible opportunities, provided that its sensible management does not fall prey to ideological division, which is already a danger. The EU may introduce its equivalent of Asimov’s three laws of robotics, but even if the US comes to the party the world’s richest man and CEO of xAI, Elon Musk has already hinted at pursuing jurisdictions with the least regulation. He posted online that "Data centers in space looking pretty appealing right now", following the November 2026 US mid-term results.
Pragmatically, it is reasonable to anticipate that AI will advance and eventually achieve super intelligence. It is equally likely that a single approach to responsible AI will not be widely adopted while competing positions have strong advocates. So idealists will be tilting at windmills trying to achieve global consensus. The lack of consensus creates a classic “prisoner’s dilemma”; any given country may accept that a slow down or guardrails are for the greater good, but still be unwilling to comply. They will be aware that if they do comply, then they may be vulnerable to another nation that, overtly or covertly, doesn’t apply the same protections. We live in a connected world, so jurisdictions with the least governance will set the pace, and their advances, and unregulated AI, may quickly disseminate to other countries.
The question then becomes whether regulation will be beneficial if implemented in some jurisdictions only or as voluntary governance. Those affected will not just be AI development companies and governments, they will be consumers, the community, and the businesses using AI. So a partial answer may be that the future is uncertain, but in uncertainty lies opportunity, so we should still take an interest and contribute to the discussion.
About the Author
Brendan has over 20 years’ experience in technology and digital innovation across the private and public sectors in Australia and internationally, working with senior leadership teams to drive strategy, operations and enterprise transformation. He welcomes professional connections on LinkedIn.