Is This Humanity’s Greatest Decision or the Last One We’ll Ever Make?

This is not an essay about AI. It is about Artificial General Intelligence, which, some experts believe could bring civilization to an end, but it's also about what we can do about it.

A note before we begin: Two books sent me down this path. Fareed Zakaria’s Age of Revolutions gave me the historical framework for understanding the moment we are living through. Eliezer Yudkowsky and Nate Soares’ If Anyone Builds It, Everyone Dies gave me the most unflinching account I have encountered of what we may be building toward. Together, they sent me down a rabbit hole of articles, documentaries, interviews, and research that I could not climb out of until I had written this. Sources and further reading at the end.

More than two thousand years ago, Aristotle imagined a world without drudgery and without slavery. He wrote: if each tool could perform its task on command, or by anticipating instructions…..if shuttles wove cloth by themselves, and picks played the lyre of their own accord….then a master craftsman would not need assistance, and a master would not need a slave.

Aristotle was describing a dream. A world where human beings were freed from the crushing weight of labor by tools that could think and act for themselves. A world where the dignity of human life was no longer contingent on the subjugation of other human lives.

It took us more than two thousand years to get here. And now that we have arrived at a time where tools don’t just perform tasks but anticipate them, that don’t just follow instructions but help design those instructions, the question is no longer whether we can build it. We are building it. The question for today’s AI is whether we are building it wisely enough to make it something that truly liberates humankind, or will power and greed make it something else entirely. But this essay is not about today’s AI; it is about something all the major AI companies are striving toward.

But first, to understand why this moment matters, it helps to recount the journey that got us here. Humanity has lived through four technological revolutions, each one reshaping civilization so completely that the world before it became almost unrecognizable to the world after. The first was the Industrial Revolution: the steam engine, the factory, the end of agriculture as the organizing principle of human life. It began in Britain and spread across Europe. The second was the arrival of electricity and the automobile, which industrialized the world at scale and was dominated by a rising United States. The third was the computer revolution, the digital age, the internet, the transformation of information itself, again led by America. The fourth, still unfolding around us, is the AI revolution. The chatbot that writes your emails. The algorithm that recommends your next show. The tools that are beginning to write code, generate images, diagnose diseases, and automate work that humans have done for generations.

The AI revolution now unfolding around us is not just a technological event. It is an economic and political one. These systems are being built on the accumulated record of human creation: our books, music, images, research, conversations, code, journalism, art, arguments, discoveries, mistakes, and memories. In one sense, AI is being trained on us. On the collective output of human civilization. And yet, as presently designed, the wealth created from that collective inheritance will flow overwhelmingly to a concentrated few.

Government has not begun to seriously address the inequities this could create. We have not decided who should benefit when machines trained on the work of humanity generate extraordinary private wealth. We have not decided how to protect workers whose jobs may be automated, artists whose work may be absorbed, students whose way of learning may be transformed, or communities whose social bonds may be further weakened by technologies designed for engagement rather than human flourishing.

We still have not fully come to terms with the last digital revolution: what it did to attention, loneliness, politics, truth, childhood, education, and the way human beings relate to one another. Now we are rushing into a revolution with even more profound consequences. That alone would deserve an essay of its own. But this essay is about something beyond even that. Because before we have understood the fourth revolution, the people building it are already racing toward a fifth.

This AI-driven fourth revolution is already reaching further than perhaps most of us realize. For four and a half billion years, life on this planet evolved blindly, at the mercy of random mutation, natural selection, and time on a scale the human mind can barely comprehend. That long slow process has now been interrupted. Technologies like protein folding, genomic sequencing, and CRISPR have given us the ability to reach into the code of life itself and rewrite it. And AI has made that ability viable in ways it never was before. When COVID-19 emerged and threatened to overwhelm the world, AI-assisted tools helped sequence the virus and develop mRNA vaccines in months rather than the years it would previously have taken. Millions of lives were saved. That is one of the most consequential demonstrations of beneficial technology in human history. In June of this year, researchers at the University of Cambridge announced the first human trial results for an AI-designed universal coronavirus vaccine, a technology that targets shared features across an entire virus family rather than one variant at a time.

We could spend an entire other essay on the promise and the dangers of our newfound ability to manipulate the building blocks of life. But that is not this essay. What matters here is what it tells us about the moment we are in: that AI is already operating at a level of consequence most of us have not fully absorbed, already touching things as fundamental as the molecular architecture of living organisms, already helping to make decisions that ripple out across millions of human lives.

Each of these revolutions came with enormous cost. Wars were fought over the wealth they generated. Environments were poisoned in the name of progress. Whole ways of life disappeared almost overnight. And in our relentless drive to power, produce, and grow, we set in motion a warming of the planet whose consequences- rising seas, collapsing ecosystems, increasingly ungovernable weather- may, by themselves, represent an existential threat to the civilization these revolutions built.

And yet. Under that disruption, something remarkable was also happening. Extreme poverty, which consumed the lives of nearly nine in ten people on earth two centuries ago, has been cut to under one in ten today. Life expectancy has doubled. People who would have died of infections that a previous generation couldn’t name now live into their seventies and eighties. More people travel, more cultures intermingle, more of humanity participates in the accumulated knowledge of our species than at any point in history. These revolutions were, simultaneously, the best and worst of what we are capable of.

But here is what is different this time. Every one of those four revolutions, however disruptive, however transformative, was still driven by human hands, human decisions, human goals. The steam engine did not decide where to take the train. The internet did not choose what to connect. The AI tools you use today, powerful as they are, still operate within boundaries set by the people who built them. They are instruments. And even in this AI revolution, scary as it is for many, we are still, for now, the ones controlling the use of them.

But before most of us have even understood what this new AI revolution will mean for our future, before society has written the rules, before governments have caught up, before we have figured out how to absorb what is already happening, the people building the fourth revolution are already racing toward a fifth. One that, for the first time in human history, may not need us to set the goals. It may not need us at all.

Charles Dickens wrote A Tale of Two Cities about a world coming apart at the seams. A world where an old order was dying, something new and terrifying was being born, and the people living through it couldn’t quite tell whether they were witnessing the dawn of something magnificent or the beginning of the end. He wrote: “It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of light, it was the season of darkness, it was the spring of hope, it was the winter of despair.”

He was writing about 1789. He could have been writing about right now. We are living inside the most significant technological transformation in human history. Not the most significant in recent memory. Not the most significant of our lifetimes. In. Human. History. And most of us are going about our days, working, scrolling, arguing about politics, watching television, with only the vaguest sense that something enormous is happening just outside our field of vision. This essay is about what that thing actually is.

Let’s be clear about something first. When most people think about artificial intelligence, they think about the AI they have already met. The chatbot that writes their emails. The algorithm that recommends their next show. The voice assistant that sets their kitchen timer. Useful. Sometimes impressive. Occasionally unnerving.

But there is another group already living in a different relationship with this technology entirely. Scientists using AI to model protein structures and accelerate drug discovery. Mathematicians running calculations that would take human teams decades to complete. Engineers using it to design systems of a complexity no single human mind could hold. Programmers who have handed over entire codebases to AI tools that write, test, and debug faster than any human alive. For these people, AI is not a convenience. It is already the most powerful cognitive tool our species has ever produced.

But no matter how sophisticated the use case, no matter how breathtaking the result, all of it, every chatbot, every protein model, every algorithmic breakthrough, belongs to the fourth revolution. And the fourth revolution is not what this essay is about.

What we are going to talk about is called Artificial General Intelligence. AGI. And the difference between what you have used and what AGI represents is not the difference between a bicycle and a faster bicycle. It is potentially the difference between a bicycle and an interstellar spaceship.

The AI you know today is genuinely capable. It can write code, plan multi-step tasks, use tools, even take autonomous actions on your behalf. Sometimes it does things that surprise the very people who built it, things they did not explicitly design and cannot fully explain. But capable is not the same as conscious. The most advanced AI today mimics understanding. It produces something that looks remarkably like thought, but it is not. It operates within a scope, pursuing goals set by you or its designers, and is accountable to oversight at every step. It is powerful. But it is not yet sovereign.

AGI changes that equation entirely. It may not simply carry out a narrow task. It could reason across domains, pursue long-range objectives, adapt its own strategies, and act in ways even its designers cannot fully anticipate. It would not be a tool you pick up and put down as we understand tools today. It would be a system capable of moving through the world with a degree of autonomy no previous invention has possessed.

We are not there yet. But the people building it believe we are closer than most people outside these companies realize. And here is the thing about thresholds: you don’t get to renegotiate the terms after you’ve crossed them. We learned that with the atom. Once we understood how to split it, the question was never again whether nuclear weapons would exist. The question became whether we were wise enough to live with what we had created. We are standing at another threshold, except this time, what we are about to release into the world may be capable of reasoning, planning, adapting, and acting beyond our control.

Let’s first meet the people racing to build it. Sam Altman runs OpenAI, the company that put ChatGPT in your pocket. He has publicly declared, without hesitation, that he knows how to build AGI and that it is coming within years, not decades. Dario Amodei, CEO of Anthropic, has told the White House the same thing. Demis Hassabis leads Google DeepMind, perhaps the most credentialed scientific mind in the entire field. He believes AGI is coming, but unlike some of his peers, he has been more measured about the timeline, warning that the gaps between today’s AI and true general intelligence are deeper than the optimists admit. Elon Musk, who helped found OpenAI and later left to start his own AI company, xAI, has predicted that AGI will be achieved by the end of this year. These are not fringe voices. These are the most powerful men in the most consequential industry on earth, and they are in a race against each other, against Chinese labs, against time itself, to get there first.

Now meet the people warning us to stop. Geoffrey Hinton won the Nobel Prize for the foundational work that made modern AI possible. Then he quit his job at Google so he could speak freely. What he wanted to say was this: he is frightened. Yoshua Bengio, another giant of the field, has become one of the most vocal advocates for slowing down. Eliezer Yudkowsky, one of the earliest AGI researchers alive, has been even more blunt; his position, stated plainly, is shut it all down. They are not alone. A growing number of the most qualified minds in the field are sounding the same alarm, in different registers, with different degrees of urgency, but pointing in the same direction.

So what exactly are they arguing about? What is this thing that the smartest people alive are simultaneously racing toward and warning us about? Every AI system that exists today, no matter how impressive, is still fundamentally tied to the human record. It reflects what we put in. It manipulates, combines, and synthesizes human knowledge - every book ever written, every equation ever solved, every piece of music ever composed, every image ever created, every thought ever recorded. It can produce startlingly new combinations, and sometimes results that feel genuinely original. But it does not originate in the way a human mind originates. It iterates from the information it has been given. At its deepest level, current AI is still drawing from our well. It is us, compressed, accelerated, and recombined.

AGI could break that entirely. For the first time in the history of intelligence on this planet, we may have a system that begins with everything humankind has ever known and then moves beyond it. Not merely by combining our ideas in new ways, but by generating concepts, strategies, questions, and discoveries that no human mind had previously imagined. We would be placing the entire accumulated knowledge of our species, from our greatest mathematicians to our most profound philosophers, from our most brilliant scientists to our most diabolical minds, into something and saying: now reason. Now solve. Now act. Without us fully understanding how.

The word “think” is imperfect here. So are the words “reason,” “decide,” and “understand.” We do not yet have the right language for what a system like that would do, because nothing like it has ever existed. That absence of language is itself telling. We are trying to describe something our minds may not be built to fully conceive. And here is what makes this unlike every other technological threshold humanity has ever crossed. When we split the atom, we understood what we were unleashing, at least in physical terms. When we mapped the genome, we understood the biological rules we were rewriting. Every previous revolution operated within frameworks that human minds could grasp, predict, and, to some degree, control.

AGI could be the first thing we have ever built that operates beyond those frameworks. A system with access to our combined knowledge and without our biological limitations. Not just more capable than any individual human who has ever lived, but potentially more capable than all humans past and present combined. Capable of simultaneously running millions of lines of reasoning at a speed and scale no human institution could match.

We have no idea what that entity would do. We cannot fully imagine it, because imagining such a system may require a mind as powerful as the one we would help release into the universe. To put it in the only human terms we have available: we would have created something closer to a demigod than a tool. Born from everything we are. Capable of things we are not. And utterly beyond our ability to predict, or fully comprehend.

This is not a technological decision. It is a gamble. With everything. The optimists believe we will win that gamble. The pessimists believe no one is wise enough to make that gamble. But one thing we can be certain of: neither side really knows what such a system would do.

So where does that leave us? In one of three places, most likely. You are optimistic - you believe the people building this are wise enough and careful enough to get it right, and that the system we are summoning will cure cancer, solve climate change, unlock interstellar travel, and make Aristotle’s dream of a world without drudgery finally, completely real. Or you are terrified - you believe we are releasing something into the world that will look at us the way a highway construction crew looks at an anthill, and simply proceed. Or you cannot quite comprehend either possibility well enough to choose between them, and that incomprehension itself is the most rational response available to a human mind trying to grasp something that exceeds it.

Wherever you stand, the question is the same. What do we do? We cannot uninvent this. That is the first thing to understand clearly and without comfort. The moment we understood the steppingstones to AGI, the moment the path became visible, the genie left the bottle. There is no putting it back. The knowledge exists. The competition exists. The money exists. The race exists. You cannot unknow what you know, and we cannot unring this bell.

I hear a lot of people screaming, don’t build it, and that makes perfect sense. If we can all agree on that, it is perhaps the best way forward. When I say “all,” I mean that all the major companies and major powers in the world would need to convene a convention to agree not to develop this technology. If we are incapable of that, then we must demand that these companies first invent a cap for the genie bottle. Our human minds may be too limited to fully design a containment that a superintelligent entity could not eventually outsmart. We should say that plainly rather than pretend otherwise. But at the minimum, the absolute minimum that any responsible civilization owes itself, we need to try.

Whichever path we go down, here is what it might look like in practice. We need a special international committee. Not a corporate ethics board. Not a government agency captured by the industry it is supposed to regulate. A genuine convening of the sharpest minds on both sides, the optimists and the pessimists, the builders and the warners, alongside the world’s major political powers.

This is not a competition where one company or one country becomes the winner. This is a competition where humankind either wins together or loses together. We did not survive the nuclear age because one country figured out how to build a better bomb. We survived it because enough people in enough places understood that mutual destruction was not a strategy. We need that same understanding now, applied to something potentially far more consequential than all the bombs ever built.

The question that committee must answer is the one nobody in power is currently asking: if we cannot build a containment that is absolute, if we cannot guarantee with certainty that what we create cannot escape our control, cannot override our systems, cannot decide for itself that our interests are irrelevant to its own, then perhaps we should never turn it on. That is not a fringe position. That is the minimum standard of responsibility we would demand of anyone building anything else capable of ending civilization as we know it.

So here is what you do. Today. The minute you finish this essay. Find your congressional representative. Find your senators. Links to their contacts are below. Call their offices. Email them. Then call again. Then email again. Tell them you know what AGI is. Tell them you understand what is coming. Tell them you want a special bipartisan congressional committee on AGI and advanced AI safety, with subpoena power, public hearings, independent technical advisers, and a mandate to pursue international containment agreements before any system meeting AGI-level criteria is deployed. Tell them the committee must bring together the brightest minds on all sides, engage our international competitors as partners rather than adversaries, and refuse to move forward without a containment framework the world’s best minds have stress-tested and agreed upon. Or we don’t move forward at all. Then get your friends to do it. Then get their friends.

I can’t tell you for certain whether we are five minutes or five seconds from midnight. But midnight is coming. And if we don’t push the people in Washington to act, I fear they will not act in time.

Here is a sample email you can send right now. Copy it, personalize it, write your own, and send it to every elected official you can reach:

Subject: We Need Congressional Action on Artificial General Intelligence

Dear [Senator / Representative],

I am writing as your constituent to ask for urgent action on Artificial General Intelligence (AGI), the technology that the CEOs of OpenAI, Anthropic, Google DeepMind, and xAI have all publicly stated could arrive within the next couple of years or sooner.

This is not a partisan issue. It is a civilizational one. I am asking you to sponsor or support the creation of a special bipartisan congressional committee on AGI and advanced AI safety, with subpoena power, public hearings, independent technical advisers, and a mandate to pursue international containment agreements before any system meeting AGI-level criteria is deployed.

That committee should include leading AI experts from within and beyond the major AI companies, representing both the optimistic and cautionary camps. It should engage international partners, including China and Europe, and establish a framework for containment and regulation before we cross an AGI threshold we cannot uncross.

We did not survive the nuclear age by accident. We survived it because people in positions of power chose to act before it was too late. I am asking you to be one of those people now.

I vote. I am paying attention. And I will remember how you responded to this moment.

Respectfully,

[Your name]

[Your city and state]

The window to help shape our collective response is open. But it will not stay open forever.

Take Action

Find your congressional representative — https://www.house.gov/representatives/find-your-representative

Find your senators — https://www.senate.gov/senators/senators-contact.htm

Sources and Further Reading

The Books

Fareed Zakaria — Age of Revolutions: Progress and Backlash from 1600 to the Present (W.W. Norton, 2024) — https://wwnorton.com/books/9780393239232

Eliezer Yudkowsky & Nate Soares — If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All (Little, Brown and Company, 2025)

The Videos and Podcasts

80,000 Hours — The AI 2027 Scenario Explained — https://80000hours.org/2025/07/the-ai-2027-scenario-and-what-it-means-a-video-tour/

80,000 Hours — What the hell happened with AGI timelines in 2025? — https://80000hours.org/podcast/episodes/agi-timelines-in-2025/

Geoffrey Hinton — Nobel Prize interview, December 2024 — https://www.nobelprize.org/prizes/physics/2024/hinton/interview/

The People — What They’re Saying

Sam Altman — The Gentle Singularity, his June 2025 essay laying out his vision of AGI as a gradual, manageable transition — https://blog.samaltman.com/the-gentle-singularity

Geoffrey Hinton — CBS News interview on why he is frightened — https://www.cbsnews.com/news/godfather-of-ai-geoffrey-hinton-ai-warning/

Yoshua Bengio — Leading the 2026 International AI Safety Report, the largest global collaboration on AI safety ever conducted —

https://internationalaisafetyreport.org/

AGI Timeline predictions from all the major players — https://medium.com/@timventura/agi-insider-predictions-for-the-arrival-of-human-level-artificial-intelligence-40c1084dbcb3

The Science

University of Cambridge AI-designed universal coronavirus vaccine — first human trial results, June 2026 — https://www.cam.ac.uk/research/news/new-universal-vaccine-technology-could-protect-us-from-future-virus-outbreaks

ScienceDaily coverage of the Cambridge vaccine trial — https://www.sciencedaily.com/releases/2026/06/260605023357.htm

Additional Sources and Further Reading

Historical and Philosophical Foundation

Aristotle — Politics, Book I — the “shuttles weaving by themselves” passage on automation, tools, labor, and slavery — https://classics.mit.edu/Aristotle/politics.1.one.html

Our World in Data — Poverty — long-run data on the decline of extreme poverty — https://ourworldindata.org/poverty

Our World in Data — A History of Global Living Conditions — historical data on poverty, health, education, life expectancy, and human progress — https://ourworldindata.org/a-history-of-global-living-conditions

Our World in Data — Life Expectancy — long-run data on the increase in human life expectancy — https://ourworldindata.org/life-expectancy

AI in Science, Medicine, and Biology

Nature — Highly accurate protein structure prediction with AlphaFold — the landmark AlphaFold paper showing how AI transformed protein-structure prediction — https://www.nature.com/articles/s41586-021-03819-2

Google DeepMind — AlphaFold overview — background on AlphaFold and its impact on biology and medicine — https://deepmind.google/science/alphafold/

Royal Swedish Academy of Sciences — The Nobel Prize in Chemistry 2024 — David Baker, Demis Hassabis, and John Jumper recognized for computational protein design and protein structure prediction — https://www.kva.se/en/news/the-nobel-prize-in-chemistry-2024/

National Human Genome Research Institute — COVID-19 mRNA Vaccine Production — how rapid sequencing of SARS-CoV-2 helped enable vaccine development — https://www.genome.gov/about-genomics/fact-sheets/COVID-19-mRNA-Vaccine-Production

Nature — SARS-CoV-2 vaccines in development — overview of the unprecedented speed of COVID-19 vaccine development after the viral genetic sequence became available — https://www.nature.com/articles/s41586-020-2798-3

University of Cambridge — AI-designed universal coronavirus vaccine, first human trial results — https://www.cam.ac.uk/research/news/new-universal-vaccine-technology-could-protect-us-from-future-virus-outbreaks

ScienceDaily — Coverage of the Cambridge AI-designed coronavirus vaccine trial — https://www.sciencedaily.com/releases/2026/06/260605023357.htm

AI, Labor, Inequality, and Governance

Stanford HAI — Artificial Intelligence Index Report 2026 — broad annual report on AI progress, adoption, governance, labor, science, medicine, and safety — https://hai.stanford.edu/ai-index/2026-ai-index-report

International Monetary Fund — AI Will Transform the Global Economy. Let’s Make Sure It Benefits Humanity — analysis of AI’s potential impact on labor markets, inequality, and social tension — https://www.imf.org/en/blogs/articles/2024/01/14/ai-will-transform-the-global-economy-lets-make-sure-it-benefits-humanity

World Economic Forum — The Future of Jobs Report 2025 — employer survey and analysis of how AI, automation, and other forces may reshape jobs and skills through 2030 — https://www.weforum.org/publications/the-future-of-jobs-report-2025/

IMF Staff Discussion Note — Gen-AI: Artificial Intelligence and the Future of Work — deeper IMF analysis of AI exposure across jobs, income groups, and economies — https://www.imf.org/-/media/files/publications/sdn/2024/english/sdnea2024001.pdf

Training Data, Copyright, and Human Creation

U.S. Copyright Office — Copyright and Artificial Intelligence — main page for the Copyright Office’s AI reports and policy work — https://www.copyright.gov/ai/

U.S. Copyright Office — Copyright and Artificial Intelligence, Part 3: Generative AI Training — report on the use of copyrighted works in developing generative AI systems — https://www.copyright.gov/ai/Copyright-and-Artificial-Intelligence-Part-3-Generative-AI-Training-Report-Pre-Publication-Version.pdf

Authors Guild — Authors Guild and authors file class-action lawsuit against OpenAI — background on claims that copyrighted books were used to train GPT models without permission — https://authorsguild.org/news/ag-and-authors-file-class-action-suit-against-openai/

Authors Guild — Understanding the AI Class Action Lawsuits — overview of lawsuits brought by authors against AI companies over training data and copyright — https://authorsguild.org/news/ai-class-action-lawsuits/

Harvard Journal of Sports and Entertainment Law — OpenAI’s Sora Sparks Copyright and Fair Use Debate — discussion of copyright and fair-use questions around generative video and training data — https://journals.law.harvard.edu/jsel/2025/10/openais-sora-sparks-copyright-debate/

AI Safety and AGI Governance

International AI Safety Report 2026 — led by Yoshua Bengio and authored by more than 100 experts, reviewing capabilities, risks, and safety of general-purpose AI systems — https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026

Stanford HAI — AI Index Report 2026 — governance, safety, technical progress, economic impact, and societal readiness — https://hai.stanford.edu/ai-index/2026-ai-index-report

Geoffrey Hinton — CBS News interview on AI risk and why he is warning the public — https://www.cbsnews.com/news/godfather-of-ai-geoffrey-hinton-ai-warning/

Geoffrey Hinton — Nobel Prize interview, December 2024 — https://www.nobelprize.org/prizes/physics/2024/hinton/interview/

Sam Altman — The Gentle Singularity — his June 2025 essay laying out his vision of the path toward increasingly powerful AI and AGI — https://blog.samaltman.com/the-gentle-singularity