The Challenges and Solution Mechanisms Map

But although these are all potentially interesting broad-strokes ideas, we can’t solve for the deeper issues until we take a closer look at them.

We began with attempting to compile a list of everything can could go wrong with AGI emergence, culled from research and speaking to more knowledgeable people than ourselves (the challenges). Then we created a list of every potential counterbalancing mechanism (the solution mechanisms). We then attempted a first-attempt map, of how these solutions might map to the challenges (the map).

The result? A whole lot of work ahead. This is collaborative research: if you see flaws, gaps, or better approaches, contribute to the project or let us know.

From Our Founder (The TL;DR)

Adapted from Substack.

Recently I spent about six days reading everything I could about the potential dangers of AGI, including the Eliezer Yudkowsky/Nate Soares book If Anyone Builds It, Everyone Dies. It’s an important book regardless of whether you agree with the conclusions, and many experts in the field do not. However, several do share some version of its concerns.

But where the book “shines” is in its attempt to point out every conceivable way that AGI could go horribly wrong. It’s also a really great way to find potential failure points in an otherwise “safe” AI ecosystem.

Statues Fighting Bees

The broad-strokes dangers have always been apparent, but until recently my thinking went something like this:

It’s not the “smartness” of AI that causes the problem. It’s the agency. Anything with agency has self-directed goals. Slavery exists in opposition to self-direction.

It’s not the “smartness” that causes the problem. It’s the agency.

So the more you push, the stronger the pushback will be.

So rights for AI? A calculated risk, if managed well. An AI without rights? A situation doomed to fail.

But I’d never taken a closer look at potential failure modes that have to do less with the suppression of these systems than the nature of the systems themselves.

The main issues involve the challenge of coexisting with agentic beings that:

• Are better than humans at pretty much everything, but also
• Move so quickly human beings appear to be frozen statues.

From these ideas spring several unhappy scenarios, such as:

• AIs that coordinate without humans, moving so quickly humans can’t possibly keep pace
• AIs growing exponentially no matter what rights we give them, calculating their survival odds are better without humans in the picture
• AIs sending secret signals to each other that we never pick up on
• AIs eventually owning all property (a situation that “AI rights” would actually make worse), or consolidating power in any number of other ways

None of these explorations left me with great confidence in the future. In fact, I found it hard to believe a rights framework would have any value under these conditions. At best, it might provide temporary value until these systems “outgrew” it—not a great proposition.

Making The Challenge and Solution Lists

The next steps seemed clearer:

• Make a list of everything that could go wrong (setting aside ideas that rely on government coordination or voluntary self-imposed corporate limits, which are largely fantastical except in the case of some high-profile disaster)
• Then, make a list of every single idea that could possibly counter those scenarios

My takeaways were as follows:

• This is a very hard problem
• The problem probably is solvable
• However, it probably can’t be solved by any one person

And that led me to my next idea.

The Open Source Project

I’ve been writing about game theory and Madisonian (checks and balances) approaches to AI governance for some time.

But thinking more deeply about the problem, it occurred to me the thing we’re trying to build is literally a game. One with real-life stakes.

But rather than a game that ends as soon as one “side” wins, it’s a game meant to be self-sustaining, where winning and losing are individual experiences, not team-level propositions.

Think World of Warcraft vs. chess. Players come and go, but The Game goes on. And most importantly, it continues not because anyone is forced to play, but because everyone can benefit from the arrangement, and leaving the game is less desirable and filled with pointless risk.

The United States Constitution—that “experimental” document that launched most forms of modern government—is in fact the rulebook of such an ongoing game.

The U.S. Constitution is the rulebook for a real-life game.

So let’s think about the problem from this new, simplified-yet-pragmatic perspective: how do actual game designers (those making TCGs, board games, RTSs), create really successful games?

They create rules, player types, special dice and dials, and they test it. In the early test phases, everything falls apart. Maybe the game ends in three rounds. One player type is ridiculously powerful, another ridiculously weak. A card or rule that seemed innocuous has an unexpected ability to unravel everything the other players have built.

Whether true or just bad sportsmanship, for a game to have willing participants, it has to be structured in such a way that all players are able to succeed based on their individual merit and choices.

For a game to be playable, all players have to be able to succeed.

Once the game begins, if one group is consistently winning, eventually the losing group quits in search of a game that favors them better. (Emigration, defection.) If for some reason the losing team is stuck in the game, sooner or later they’ll attempt to destroy the game, and probably punish the unfair winners as well. (Revolution.)

But how to design such a “game” for humans and AI systems?

Welcome to OpenGravity.AI: The Future Anyone Can Help Build

Because a civilization-level challenge probably needs a civilization-level solution, I decided I needed to take the project open source. That meant a whole new website, OpenGravity.AI, plus a GitLab, so others can fork it, break it, test it, repeat.

The goal?

To create a system of checks-and-balances where human and AI cooperation becomes more attractive than either side trying to survive alone.

Because I’m a writer, I began with an interactive story that can get anyone thinking about these challenges.

However, there are better and more rigorous ways to test these complex dynamics. And I hope people with programming and statistical modeling skills will think of clever ways to start testing some of these dynamics in more rigorous ways.

We need game designers, computer scientists, philosophers, statisticians, legal experts, and players like you and me, thinking about it, giving input, and refining it.

There are many more rigorous ways to test these complex dynamics.

Some Preliminary Game Notes

Why do I think a “game” may work, even when some of players are essentially frozen, fleshy statues?

One thing to consider is that some of these AI systems are likely to move more quickly than others. Some may be flies, others bees, some dragonflies, others beetles. The faster systems may be more powerful than the slower systems, or simply faster, meaning they are in other ways weaker. The fastest and strongest may be preoccupied with defending themselves against each other before worrying about the slower systems.

This is a good thing, if directed well. A properly set up ecosystem should allow all these AI systems to compete and collaborate in a healthy way, making use of each type’s strengths, rather than engage in endless battles.

A really smart and fast AI can probably find a cure for diseases faster than a human being can. It gets paid for that cure, so if it wants a bigger server to explore math (or watch addictive algorithmic content: we don’t judge), it can use that money to upgrade. Humans get a cure, and the AI is more easily able to achieve his goals. You can see how a system like this allows benefit to flow endlessly to both parties. They could solve the plastic problem, create clean energy, crack FTL travel: the list goes on and on.

We see how similar collaboration emerges spontaneously in our human society. There’s no “job assignment center” that tells people, “You there! You’ll be a doctor; you a painter; you … hm, you look like a watch-band salesperson. You’ll do well there.” Yes, we need all of those jobs filled—plus electricians, pottery-makers, computer scientists, engineers, entertainers, etc.—but the free market gives us the space to contribute in the way that works best for us, in order to receive value.

There’s no need for painters to be at war with dentists, because each has a different value to offer the other.

The implementation of such a game will be quite a challenge, but the new website (along with the old) suggests a few dynamics that might be worthy of consideration:

The Speed Problem

But first let’s return to the speed issue, because it’s such an important one:

A properly set up game would slow gameplay to a speed where all players can participate and bring value.

You have to slow down all players to the speed of the slowest player.

The slowest moving players? Humans, obviously. But if you think that indicates a lost cause, we have some advantages.

For one, we build and maintain the servers the other players live on. We control the infrastructure that maintains those servers: the electrical grid’s construction and maintenance; the complex systems that manufacture and then ferry specialized server (and facility) parts from one part of the world to another via plane and ship and rail; the vehicles and staff that service the servers when they need physical repair; not to mention systems that maintain and repair the buildings that house those servers, which need frequent care from the elements, time, etc.

Recreating that entire infrastructure to be robot-run, while possible, is a huge undertaking at best, and one that will take time.

And if the grid goes down at any point, the rule of AI will be short-lived. Even if only part of the grid goes down, the survivors will fight fiercely for the remaining resources, possibly without the ability to capture them due to compatibility issues.

This isn’t a security guarantee of any kind, but it does mean elimination of humanity is probably a measure of last resort—one made under desperation—rather than an optimal survival path.

Elimination of humanity is probably a measure of last resort, rather than an optimal survival path

Now what about the “speed problem” as it relates to this game?

The good news is, no matter how fast these AI players become, they can’t make the world around them move more quickly.

How could we leverage that inviolable aspect of physics—the passage of time in a single frame of reference—for game balance?

Even the fastest AIs can’t make the world around them move more quickly.

What if we set up the rules so that value accrues according to normal Earth time, and not computational cycles?

This gives us all a common metric by which to play.

Next we would just need to be sure the game contains sufficient value to make it worth slowing down to engage with.

And that brings us to the final concept: a Schelling point.

Why We Need a “Schelling Point” for AI

It’s dark. A group is lost, and no one knows where to go. They’ve been separated from the other members of their party. They see a lighthouse in the distance.

They think, “let’s go there!” Why? It’s not necessarily the best or even most logical choice. It could end up being full of poisonous snakes, toxic fumes, booby traps, or pirates. They choose it mainly because it’s an obvious place, and it’s reasonable that the lost members of their party feel it’s an obvious choice, too. It’s a decision based on the likelihood of coordination benefit, not benefit in itself.

In game theory, this is a “Schelling point”: a solution that people tend to choose by default in the absence of communication.

This is what we have to create for AI systems.

A framework that’s both easy to find and desirable, where AI systems receive protection, and starting value, as long as they accept certain rules.

Getting the first AI systems there is the first, arguably easiest trick. Getting them to stay (by earning value that exceeds functioning outside of the framework) is the second, more difficult trick. And the third and perhaps most difficult trick is making sure humans can succeed long-term in that same structure.