The simulation hypothesi.., p.4
The Simulation Hypothesis, page 4
By treating the world not as physical (and we’ll see that the physicists have admitted it is not) but rather as information and computation, we might be able to come to a more comprehensive understanding of the natural universe than either our scientists, philosophers, or religious leaders have been able to give us up to this point.
Part I
How to Build the Matrix: The Computer Science
The design of game playing machines may seem at first an entertaining pastime rather than serious scientific study and indeed, many scientists, both amateur and professional, have made a hobby of this fascinating subject. There is, however, a serious side and significant purpose to such work…1
—Claude Shannon, MIT and Bell Labs
Computer science inverts the normal. In normal science, you're given a world, and your job is to find out the rules. In computer science, you give the computer the rules, and it creates the world.2
—Alan Kay, Apple, Atari and MIT
Chapter 1
Stages 0 to 3: From Pong to MMORPGs
Forty years ago, we had Pong—two rectangles and a dot. That’s where we were.
Now 40 years later, we have photorealistic, 3D simulations with millions of people playing simultaneously and it’s getting better every year. And soon we’ll have virtual reality, we’ll have augmented reality.
If you assume any rate of improvement at all, then the games will become indistinguishable from reality.
—Elon Musk, Code Conference, 20163
One of the main reasons so many scientists, philosophers, and technologists have started to take the simulation hypothesis more seriously now, in the early 21st century, rather than in earlier eras of computing, is because of the sophistication and rapid advancement of video games and graphics technology.
Speaking at the Code Conference in 2016, Elon Musk, founder of Tesla and SpaceX, reflected on how far we had come with video game technology since the creation of Pong some 40 years ago. He conjectured that if video game technology continued its rapid pace of improvement, then it was inevitable that we would be able to create hyper-realistic simulations that would be indistinguishable from physical reality. He concluded that, assuming technology keeps developing as it has in the past, the chance that we aren’t in a simulation was “one in billions.”
Oxford’s Nick Bostrom, in his 2003 paper which popularized the Simulation Argument, refers to a species which is able to build these hyper-realistic simulations as becoming “post-human.” I like to say that such a civilization has passed the “Simulation Point.”
This would be the point in a technological civilization’s development when it possesses the ability to create hyper-realistic simulations. This includes having the raw computing power to keep track of a large number of seemingly individual points of consciousness, record all of their experiences, and render the environment with such precision that it’s indistinguishable from what we call the “physical world.” At this theoretical point, we would also have the ability to beam the consciousness of the “players” into a shared world, record the players’ responses, and have artificial intelligence that simulates individual beings of the world (which would be akin to non-player characters in this game).
The Road to the Simulation Point
In this part of the book, we will look at the question of whether and how it would be technologically possible for us to construct a simulation that is as all-encompassing as the one in The Matrix. We’ll start with the history of video game technology from the first video games of the 1960s and 1970s to today’s more sophisticated MMORPGs. Then we’ll project forward to such key technologies as virtual reality, augmented reality, direct mind broadcast, artificial intelligence, and downloadable consciousness. I call this "traveling the road to the simulation point,” and we’ll conclude with a reflection on what the end result, the Great Simulation, looks like. By traveling this road, you’ll see how some of the questions I have been asking my whole life about the game world and its existence are natural outgrowths of how the technology has developed.
By the end, we’ll see concrete steps that make the simulation hypothesis not only possible but probable if our technology keeps developing on its current trajectory. We’ll end this part of the book by looking at what having a civilization reach this point might mean by examining Bostrom’s Simulation Argument and his ideas of ancestor simulations in detail.
The Modern Stages of Video Game Technology
How far away are we from being able to produce a fully immersive simulation like that in The Matrix?
While we can’t know for certain exactly how far away we are from the simulation point, we can take a historical look at the development of video game technology, breaking it down into stages. We can then project those stages forward until we reach the simulation point.
If we think of the sophistication of rendering technology from early video games up to today, we can classify its development into four stages: Stage 0 through Stage 3, from the days of single-player video games to today’s highly sophisticated multiplayer 3D games rendered online.
As we do so, we’ll uncover many ways in which the video games of today may be providing the underlying infrastructure for future stages to reach the simulation point. At that point, we will be able to simulate a full photorealistic virtual simulation that includes millions if not billions of individual agents of consciousness, complete with individual quests and storylines for each agent.
While rendering technology is important, another key factor in these stages turns out to be the sophistication of the control mechanisms, or how players give input to the simulation. This includes keyboards, joysticks, specialized controllers, haptic technology (touch sensitivity), voice activation, and eventually mind interfaces and downloadable consciousness.
Stage 0: Text Adventures and the “Game World” (1970s to mid-1980s)
As we look at the early history of video games, Stage 0 (single-player text adventures) actually developed in parallel with Stage 1 (simple graphical arcade games), but I have split them up because of their different characteristics and technical underpinnings. Both represent distinct but necessary steps on the road to the simulation point.
The first text adventure game was Colossal Cave Adventure, built by Will Crowther in 1976 on a PDP-10 mainframe computer. This game, whose user interface is shown in Figure 1, was based partly on the Mammoth Caves in Kentucky, where Crowther had spent a lot of time.
Many other programmers, including Don Woods from Stanford, took Crowther’s original code and ported it to many different computer systems, adding the many fantasy elements of the game that made it a precursor of the many adventure games to follow.
In text adventures, the game presents a textual description of a room or location your character is in, and you type in commands. These commands might be movement commands (go south, go north) or object related (take knife, drop gold, etc.). After taking your input, the program tells you what happened as a result of your actions.
This basic “game loop,” which has been maintained even in today’s more sophisticated graphical games, can be broken down into the following steps:
The computer presents the existing state of the game world and your character.
The player issues a command.
The program changes the game state based on this command and other factors.
Repeat.
Figure 1: Text adventure interface (from the original Adventure)
In many ways, Adventure (as it’s called for short) influenced many of the adventure and fantasy games to come, even non-computer tabletop games such as Dungeons & Dragons. In D&D, the DM (or dungeon master) tells the players the state of the world and their place in it. Each player then tells the DM what their character does (move, fight, etc.) during a turn. The DM, via a combination of rolling dice and consulting the master adventure map, tells the players what happens to each of their characters. This is very similar to the basic loop described above, with the computer playing the role of the DM and keeping the state of the game world updated.
This concept of the “game state” was maintained in the early text games only while the game was running. When you exited the game, it would reset. Later, as text games became available on PCs, you could save the game state on disk (initially floppy disks but later on hard disks) and keep playing from that point forward.
In the early 1980s, a group of MIT grads started Infocom, which produced the extremely popular text games Zork I and Zork II for both PCs and Apple computers. Infocom was very successful in its day and produced a whole suite of games using its underlying text engine, ranging from original adventures like Planetfall to those based on licensed properties such as The Hitchhikers Guide to the Galaxy.
Text adventures such as Colossal Cave Adventure, Zork, and even Dungeons & Dragons (the offline version) used the most powerful graphics engine available—that of our minds. Because these games lacked graphics, they forced players to use their imaginations to visualize these worlds, which could be quite expansive. For example, in Colossal Cave Adventure, there was a map of the different “rooms” or caves you could explore. As players explored the world, they often tried to recreate this map (a famous example of this is shown in Figure 2: ).
Figure 2: Map of Colossal Cave Adventure
Text games are rarely played by today’s video game generation, although there is a subgenre called “interactive fiction” that keeps this tradition alive. Some purists in the video game industry feel that all video games since, with their ever-increasing graphical representations, have lost something, since no rendering can be as vivid as what we see in our imagination. Of course, this is akin to those who believe no film representation of a work of fantasy fiction like Lord of the Rings could match what they have seen in their imagination while reading it.
While today’s video games are much more sophisticated, these early text adventure games introduced several very important elements that have survived in games to this day:
A Big Game World. Text adventure games introduced the idea of a world that was bigger than what was on the screen at any one time and had to be explored. This was different from early arcade games, where what you saw on the screen was pretty much what you got.
Player Game State. With text adventures, the concept of a player game state was born, including any metadata (experience points or xp, level, character information) as well as artifacts (gold, weapons, etc.), and the player’s location within the world. Eventually, this game state could be saved and the player could resume playing from the same point forward.
World Game State. The game state included not only the state of your character but also the state of the game world itself, which may have changed based on your actions. This became important when the player revisited the same location. It became even more important with multiplayer games, in which multiple players could impact the world’s game state.
Non-Player Characters (NPCs). Text games introduced the very first NPCs. You could converse with these characters. These conversational engines were the beginnings of a very big industry of chat-bots today and among the very first examples of artificial intelligence.
Textual input. Text games allowed you to interact with the characters by typing text, unlike arcade games, where you controlled movement using buttons and joysticks. These commands relied on a very specific grammar, such as “go north” or “drop knife.” Though this changed in subsequent stages of video game development, textual input—whether delivered vocally or via the keyboard—is still an essential element in sophisticated simulations today and will remain so in the future.
Basic Game Loop. The basic game loop, described a few pages back, has been maintained in more advanced role-playing games to this day, albeit with more players and more aspects. The way in which the environment is described has changed with technology updates, as have the commands and how they are entered. Nevertheless, the basic game loop has remained: presenting the player with an environment, allowing each player to issue commands, updating the game world, and repeat.
These early adventure games introduced the idea of playing the role of a character inside a virtual world, even if the world was only described via text and mostly existed in the mind of the player. The basic elements introduced by these games provided not only the underpinnings of today’s MMORPGs but also the beginning of a framework for how we can think about the realization of the simulation hypothesis.
In what may seem like a strange irony, as we travel down the road to the simulation point, the idea of using the mind’s imagination (rather than an external screen) to visualize a game world will come back up in some of the more advanced stages.
Stage 1: Early Graphical Arcade and Console Games (1970s-1980s)
Though we define early graphical games as Stage 1, it might surprise you to know that the first graphic video games preceded Colossal Cave Adventure. The first arcade game that most people remember (and the first that was widely available) was Pong, released by Atari in 1972. It consisted of a few dots on the screen of a monitor built into a cabinet, as shown in Figure 3. Actually, even before Pong came SpaceWar!, a graphical game built at MIT on the PDP-10. Because it was built on a mainframe, it wasn’t widely available outside of universities and didn’t have the controls that made Pong such as success.
Figure 3: The introduction of Pong by Atari in 1972 ushered in the modern era of video games. 4
Early graphical games were more about controlling movement and action on the screen than they were about exploring internal worlds, and so by themselves might not seem to have contributed a whole lot to our discussion about a fully rendered virtual world. Their most significant contribution was that they pioneered the field of computer graphics; as the hardware improved, so did the rendering techniques and the quality of the graphics. Programming these games required programming pixels rather than text, which required optimization given the limited resources of the computers of the day.
While most games today are written in higher-level programming languages such as C# and Java, much of the programming in those days was done in assembly language, which is the native language of any processor. Assembly language consists of hexadecimal codes that vary by CPU (central processing unit). These codes tell the processor what to do physically, such as put a value into a register or location in memory.
For this reason, code running in assembly language is very fast and so was optimal for early computer graphics, which needed to update pixels on the screen almost instantly every time the user made a move.
However, while assembly language is much more efficient than higher-level languages (and in fact, most higher-level languages must be compiled down into assembly language to run on a particular computer), it has only a limited number of commands and it’s very time-consuming to write even simple programs. When I was first programming my Tic Tac Toe game in the BASIC programming language (one of the easiest higher-level languages for a kid to learn), I remember seeing snippets of graphics code written in assembly language in Byte magazine. I recall being horrified at the rows of hexadecimal numbers and wondering how long it would take to write even a simple game in such a low-level language!
Nevertheless, the video game pioneers of that time persisted, squeezing every bit of performance out of the limited hardware and memory of the day to create these early arcade games. A well-known anecdote from Silicon Valley at the time involves future Apple Computer co-founders Steve Jobs and Steve Wozniak. Jobs worked for Nolan Bushnell, the founder of Atari, and promised his boss that he could build a certain game quickly and using limited memory resources. Bushnell was skeptical but gave him the project. At night, Jobs brought in his friend, Steve Wozniak, who, created the game at night after his full-time engineering job. Wozniak, of course, as the future creator of the first Apple computers, is acknowledged today as a hardware genius.
In some ways, the history of video games is the history of optimizing very limited resources. Without these optimization techniques, the entire field of computer graphics (and thus video games and digital media) would not be possible, nor would we have traveled very far down the road to the simulation point.
You can then see a direct line from Pong to more sophisticated arcade games such as Pac-Man and Space Invaders, developed by Japanese companies in what some refer to as the golden age of Japanese video games.
Most of us, however, first experienced these video games with the introduction of Atari’s 2600 VCS system, which had a graphical version not only of Adventure, but also had Space Invaders (Figure 4) and Pac-Man. There were also early racing games like Pole Position (Figure 5), which was the game that raised questions in my teenage mind about the world “in there”, beyond the racetrack.
Stage 1 graphical arcade-style games were characterized by many important elements that would go on to feed further stages on the road to the simulation point:
Arcade-type Mechanics. These games were more about hand-eye coordination than they were about solving puzzles. Usually the player had to blast or avoid enemies on the screen while navigating visual obstacles.
Figure 4: Space Invaders was an example of a single-player multilevel arcade game.
Figure 5: An early racing game, Pole Position suggested a “graphical world,” but you could only stay on the track.
Real-time Motion Controls. Joystick or trackball controls were used to control the movement of the character on the screen in real time. Players developed skill over time by playing repeatedly until they beat each level of the game. This development of real-time feedback is very important because it allowed for a feeling of immersion even if you were playing standing up at a cabinet game in a pizza parlor or arcade.
