Chapter 1: Introduction
Wherein I introduce the idea that we might look to neuroscience research for a surprising way to interpret modern AI programs—as analogs to brain cortexes
Note: This chapter is part of the proof-of-concept material for a book, AI: How We Got Here—A Neuroscience Perspective. If you like it, please consider supporting the Kickstarter project.
1.1. Stepping Stones
I first spoke with the neuroscientist Stephen Larson in December of 2022. He was a modest man, in his early forties, with an expression that could be sometimes boyish, sometimes gloomy. Over the course of four separate conversations, I learnt about his research, and how it had led him on his own particular journey to try to understand the mind, one that many scientists undertake—albeit usually in less spectacular fashion.
Beginning in 2010, Larson had embarked on one of the first independent efforts to create a realistic computer simulation of a living organism, a member of the tiny, one millimeter-long species of worm known as Caenorhabditis elegans, or more often, just C. elegans. He and the collaborators that would join him aspired to make a computer version of a worm—in a computer-simulated environment—that would do everything the real worm did, like crawl, forage, and even mate with other worms, which (who?) were also being simulated.
To do this was a way of trying to figure out exactly how the worm works—and especially, the mind inside it. The first version of the simulation would inevitably be imperfect, perhaps not even crawling, or failing to demonstrate all the worm behaviors that were expected. But by comparing it with reality, you could figure out how to make improvements. The second version of the simulation would inevitably still be imperfect, but hopefully, a little more realistic. By continuing through many iterations of this process, the idea was that you would eventually pin down exactly how to build a realistic computer simulation of the worm, and exactly how the worm worked.
This endeavor came to be known as the OpenWorm project, styling itself as a volunteer exercise in open source software development. It came to a culmination in 2018, when Larson and his collaborators were invited to present their results at a special meeting of worm scientists at the Royal Society of London, one of science’s most hallowed institutions, where figures like Isaac Newton once presided. “It was a dream,” said Larson. “We’d gotten not just press support, but legit academic support from well known scientists."
But by that time, the best simulation that Larson and collaborators could come up with was a long ways away from realistic.1 It could crawl, yes, in a computer-simulated environment, yes, but it was a simulation more in ambition than it was in practice; really, it was more like a puppet, with strings that had to be animated by the scientists. It was so far away from being realistic, in fact, and so roadblocked, that the project had effectively met its sink-or-swim moment, which for various reasons, would soon submerge them.
A cynic might conclude that the OpenWorm project was a failure, or that it was wasted effort. However, failure is the status quo of science. For most of human history, it has not been possible to understand the world around us, let alone the world inside us, which is so much more delicate and difficult to work with. Trying to do so can literally kill us. Larson’s journey was therefore not distinguished as a failure. It was distinguished as a starting point, an attempt to take on one of the biggest challenges in all of science, the challenge of understanding the mind (or a mind), without getting sidetracked. It was an example of bravery in science.
But why try to simulate a tiny worm, in the first place—isn’t it the human mind that neuroscientists chiefly care about? It is true that Larson was not above all concerned about the worm, like so many other biologists and neuroscientists who work with worms, or mice, or monkeys. For him, the worm mainly played the role of being a stepping stone to progress. If you could understand the mind of the worm, then it would seem to provide a huge leg up on understanding the minds of humans.
The strategy that Larson undertook to understand the mind of the worm, the strategy of simulation, was itself just meant to be another stepping stone to progress. Consider that whenever scientists want to understand anything, such as why three slices of pie disappear from the office every day, in an office with three workers, they must start by making a hypothesis. Over time, they will make that hypothesis increasingly precise and quantitative, putting it into the form of mathematics, for example, using mathematical fractions to represent the pie slices. Gradually, they might put their hypothesis into the form of equations or statistics, so that they could precisely represent the entirety of the pie-disappearing process. By then, their hypotheses start more commonly being known as models of the situation. And as they make those models more and more realistic and comprehensive, they then shift to thinking of them as simulations. In other words, building models and simulations is the conventional scientific method. It is the route that all scientists always take whenever they want to get answers.
It is true, however, that simulations play a sort of special role in science. For one thing, they are intuitive. Because if you can build a realistic simulation of what something is, whether as a real-life, wood-and-metal (or bone-and-tissue) reconstruction, or more commonly, as a mathematical equation or computer program, then it must mean that you can understand it—otherwise, how else would you have been able to build the simulation, in the first place? A variant of this observation was immortalized on the influential physicist Richard Feynmann’s chalkboard when he died, repeated so often since that it has become a tired, yet enduring cliche of science: “What I cannot create, I do not understand.”
So it was that from Larson’s perspective, simulation was to take us to the mind of the worm, and the mind of the worm to take us to mind of the human. Unfortunately, even with this helpful set of stepping stones, the sorts of questions that Larson wondered about—whether in regards to the worms or humans—still confound us.
Indeed, to understand the human mind—and how it arises from the brain beneath it—remains arguably the most important goal in all of science. Today, we are still far away from having achieved it, and that’s a problem. We have only limited abilities to treat the ailments that we suffer from so badly, like brain injuries and mental illnesses. You only need to visit a local hospital to appreciate the desperateness of our ignorance. The mind is a grand mystery, but nothing like a black hole, which placidly beckons our curiosity from a distance. The mind is the cosmic enigma that sits right in front us, speaking and smiling, yes, but also crying, screaming out at us.
1.2. Artificial Cortexes
Larson’s journey amply demonstrates how building minds (in the form of simulations) has historically proved impossible. But remarkably, over the last decade or so, neuroscientists have started to make dramatic and unexpected progress. Even more surprisingly, they have made it in large part by jumping over the worm, or other organisms, and diving straight into the minds of humans.
The roots of this progress can be traced back to 2014, with the work of those like James DiCarlo, a neuroscientist at the Massachusetts Institute of Technology (MIT). I spoke to him in the spring of 2023, scheduling a chat with him through his laboratory manager, who coordinates the work of a small cottage industry of researchers. DiCarlo was bright and animated when we spoke, wearing a New Balance pullover, confident in the epoch-making quality of his work, but also quick to acknowledge its gaps and shortfalls.
Along with his students and collaborators, DiCarlo discovered for the first time how to create a compelling model of the region of the human brain known as the visual cortex, which like the name suggests, is responsible for our remarkable abilities with vision.2 His model, made from equations, served as a hypothesis for how the visual cortex works, and how it’s operated.
Neuroscientists had long made models of the visual cortex, but DiCarlo’s model of 2014 was singular for two central reasons. First, it could perform some of the most powerful behaviors of the visual cortex, like recognizing familiar objects, something neuroscientists had never previously been able to make models accomplish. Second, it did so using internal patterns of signals that were closely correlated with the real signals measured from the real mammalian visual cortex, while it was also engaged in recognizing objects. With these two characteristics together, the model could be said to perform the same visual tasks as humans in a similar way as humans, thereby providing a first compelling explanation of how this brain region functioned.
But to better explain how such a model could be created, consider a more recent study, produced by a team of neuroscientists in March 2023.3 The senior author is Srinivas Turaga, of the Janelia Research Campus in Ashburn, Virginia. He has a wry smile, and a good sense of irony for how the models of neuroscientists can be at once so beautiful, but also so simplistic, like lego-block mockups of a starship.
His study focused on the fruit fly, a tiny insect with a head and brain-like region in it. This fruit fly brain is made up of many tiny biological compartments called cells, some of the most important of which are known as neurons. Neurons are known to be connected to each other in intricate patterns, a little like the stations of a grand subway system. Bioelectrical signals constantly flow through these stations, giving rise to all the physical and cognitive functions of the organism.
“Let’s say I’m going to swat a fly,” explained Turaga. “It needs to see this very quickly, and then respond to jump away.”
The region of the fruit fly’s brain that allows it to detect this motion is called its visual cortex (sharing the terminology with humans). It is directly connected to the fruit fly’s eyes, which transmit signals through other neurons into it. It has been studied for decades, while for the most part never even being considered as something to be simulated. To do so would have always seemed impractical or overambitious. But in their study, the authors present arguably the first such model of the fruit fly visual cortex worthy of being called that.
How did they do it? Well, for every tiny neuron in the visual cortex and the eyes, the researcher created a sub-model of that cell in the form of a mathematical equation, just like using a fraction to represent a pie slice. They then connected those sub-models together, using more equations, in a way that faithfully represented the way the cells are known to really be connected. Lastly, and perhaps most importantly, they optimized the strengths of these neuron-to-neuron connections, represented as numbers in the equations, in the same way as the real connection strengths are thought to have been optimized by evolution—for detecting motions.
After constructing a composite model of the fruit fly’s visual cortex in this way, basically, a big bird’s nest of equations, the researchers then put it into the form of a computer simulation. (How they do this is relatively routine, and something we will come to shortly.) This simulation then showed that the neuron models (often called artificial neurons) generated (models of) signals in response to sudden motions in the same way as real fly brain neurons have been measured to respond to sudden motions, in real experiments.
The results that they present suggest that the researchers have successfully built a simulation of a biological brain region. It is a simulation that you might describe as semi-realistic, somewhere between realism and unrealism. It is far from being realistic in every way—something that researchers see as self-evident. For example, their model does not produce signals with exactly the same strengths as signals that have been measured in experiments; and their model of the fruit fly visual cortex is not integrated with a model of the rest of the fruit fly brain, making it strangely isolated. Indeed, if one were to comprehensively list all the ways that the model differed from reality, the list would be long and prominent.
Nevertheless, it is inarguable that this model reproduces some of the most important microscopic and macroscopic features of the fruit fly visual cortex. It detects motions, and it does so using similar signals, sent around in similar structural patterns, as the real fruit fly visual cortex. It is exactly the sort of thing we think about, generally speaking, when we think of making simulations of brain regions. The authors use the word ‘simulation’ eleven times in their paper.
1.3. Hints of Revolution
Turaga’s work serves as a prime example of how since 2014, neuroscientists have quietly begun succeeding in the quest to build brain simulations—or at least, simulations of brain regions. Although Turaga’s study focused on the fruit fly, many other studies focus on creating simulations of human or mammalian brain regions. These include the visual cortex, as aforementioned, but also very different regions, like the so-called ‘language network,’ the region of the human brain known to be responsible for basic language functions, like language processing, comprehension, and generation—functions unique to humans.
It is broadly agreed by neuroscientists that these models are of great significance. They do what the brain regions do in terms of macroscopic function; for example, recognizing objects, or generating coherent language. But they also serve as plausible models of how the brain accomplishes these tasks; the mocked up signals inside their mocked up structures are significantly correlated with real brain signals in real brain structures. They therefore provide a compelling explanation of how the brain gives rise to the mind, the central question of neuroscience.
However, there is something fascinating going on in that neuroscientists do not tend to call these models simulations (Turaga’s study being one exception). This is for many reasons; one of them is that they are uncertain of their realism. Which is to say, neuroscientists acknowledge their models to be realistic in certain ways, but there are also many ways, as we have pointed out, in which they know them to be patently unrealistic. By stopping at calling their creations ‘models,’ neuroscientists can hedge about the realism of what they’re creating.
At the same time, if neuroscientists were merely creating models of the mind, then they would be doing no more than what they had been doing for the last century. But that would be to underappreciate what they’re doing; arguably, it would be a wholly inaccurate interpretation. Studies like Turaga’s show that there is a real break with what they are doing now from what they had ever before been doing. In many cases, they are making the first predictive models of brain regions—for example, the first models to take an input similar to a human (or a fly), like an image, and reproduce the brain’s microscopic and macroscopic functions.
So, while there might be a good deal of justified hand wringing amongst neuroscientists about whether or not they are making realistic models of brain regions, regardless of what you call them, we are entering what seems to be a period of scientific revolution. We are finally beginning to decode the cosmic enigma of the mind, by executing on the strategy articulated by those like Feynman. We are creating vivid reproductions of large-scale regions of the brain for the first time in history.
That is wonderful. But in so doing, our fundamental understanding of nature, humanity, and the cosmos is also changing. The field of neuroscience is undergoing a seismic shift, from seeing the brain as an arcane mechanical object of almost unspeakable complexity, to seeing it as something explained remarkably well as a product of optimization. For example, the fruit fly visual cortex simulation that Turaga and his his co-authors created relies mainly, for its main ingredient, on the standard application of a neuron connection strength optimization algorithm. Who knew that something so algorithmically straightforward could be so closely related to a brain region.
Understandably, for many scientists, this intellectual shift is a little unsettling. Indeed, it seems a primary reason that neuroscientists might be hesitant to call their models simulations is simply because they have discovered that the nature of what it means to be a brain (or brain region) is simply so much different than what they previously expected. Stated differently, when they struggle to interpret their progress as progress in brain simulation, it may have more to do with spectacular nature of that progress, rather than any scientific pedanticism.
But in the past, there have been many scientific revolutions. And if one takes a historical view, then it is always the case during these periods of transition, such as the establishment of the particle nature of gasses, by Ludwig Boltzmann, or the discovery of quantum mechanics, as a theory of atoms, that scientists are forced into uncomfortable positions. This suggests that we would perhaps best be served by trying to sit with the discomfort of neuroscientists, rather than reject it; their discomfort that is reflected in many ways, but especially in the way they are hesitant to think of their models as simulations.
Undoubtedly, the single greatest discomfort suggested by all their progress, the most important one to sit with, is the way that it has exposed deep links between brain regions and artificial intelligence (AI) programs. Because a startling fact of all of these new ‘would-be, might-be’ brain region simulations, like Turaga’s, is that they are made in very similar, or even identical ways to modern AI programs. They are fundamentally similar in construction to programs like ChatGPT, the chatbot released by the company OpenAI in November 2022, which has rapidly become a household name to the public.
And just like other household names of commercial industry, like a Volkswagen vehicle, or a Whirlpool dishwasher, or Microsoft Office program, we tend to take it for granted that these things—even when very sophisticated—are nothing like real brains, the core of what makes us human. But if neuroscientists are now beginning to succeed in building brain regions out of AI programs, as the evidence suggests, then that also suggests that AI scientists, who are building fundamentally similar programs, are also making technology products that are closely related to real brain regions.
In short, as radical it may sound, our technology is becoming human. And that is not an offhand statement; the evidence for it is rigorous, printed openly in the journals of biological scientists, and steadily mounting.
1.4. The Simulation Interpretation
I call this way of seeing AI programs—seeing them as directly analogous to simulations of brain regions—the simulation interpretation of them. It provides an intuitive but also scientifically rigorous interpretation of AI that is badly needed, in a world where powerful AI programs are now being so rapidly developed, deployed, and propagated. It provides an elegant way to understand the strengths and weaknesses of modern AI programs, like ChatGPT; surprisingly powerful, because of its rigorous correspondence to brain regions, but seriously limited, because it so far mainly corresponds only to the mind’s fragments.
As valuable as it may be, this simulation interpretation of AI—and more generally, seeing progress in neuroscience as progress in brain simulation—is only valuable insofar as it is justified and accurate. A core purpose of this book is therefore to follow along with the neuroscientists who have uncovered the evidence both for it and against it, which has gradually come forth over the last decade.
The reasons for and against the interpretation take many forms; they are technical, sociological, and philosophical in nature. They are fascinating, in their own right, like a sparkling castle moat that is just as beautiful as the castle, at its center. But to truly appreciate the progress in neuroscience, we must cross over this interpretive barrier. After all, we cannot assess the results that scientists have uncovered if we cannot interpret them and assess them in plain language, which is precisely what the language of simulation offers.
Are AI scientists now making computer programs that are directly analogous to real brain regions, and does that explain their remarkable strengths and weaknesses? Or are their similarities merely coincidences, with AI programs and minds having nothing but superficial resemblances in common? These are the sorts of questions I seek to examine.
1.5. Brain Schematics
This book will delve deep into the remarkable AI programs that both AI scientists and neuroscientists are now creating. But to understand how an AI program could be so closely related to a brain region, it is helpful to first understand what an AI program is, fundamentally, and how it differs from a conventional computer program.
In a conventional computer program, like a word processing program, every aspect of the program that a user interacts with, like a menu with a set of dropdown buttons, is generated from a text file of human-written instructions. Programmers write these instructions (often just called code) in what is known as a programming language (such as C, or Javascript, or Python). This is a set of commands designed to be interpretable by both humans and computers.
In contrast, a modern AI program is a very different kind of program. It is inherently a little similar to what scientists tend to think of as a computer simulation program. But to get a sense of what a computer simulation is, in general, let us first consider a simple and ubiquitous example, where a weather scientist wants to make a computer simulation of the weather, which would allow them to make weather predictions.
To do this, they would first look up a mathematical model known to represent the flow of air and water. For example, they might open a fluid dynamics textbook and find the well-known model known as the Navier-Stokes equations. These equations were invented in the nineteenth century, and have been established over the decades to faithfully represent the flow of liquids, even in very complex situations, such as fluid turbulence.
The mathematical purpose of these equations, just like all equations, would be that you could solve them. More specifically, you could solve them for certain variables of interest, like the velocity of the air, over a certain timespan—as long as you specified certain input data, like the air velocity at the beginning of that timespan. The equations, the data, and their solution—in their totality—are often said to describe the velocity’s time evolution.
But let’s go back to practicalities of our simulation creator. After assembling these equations and the data needed to solve them, the weather scientist would then write those equations into a computer program. Next, they would write the code needed to automate the process of solving the equations. Finally, they would run the program, solving the equations, and thereby gain a prediction for the airflow at some time in the future; precisely the information needed to make weather predictions.
A modern AI program of the sort we consider in this story is also a program that solves equations. However, these equations that an AI program is based around are much different than most equations encountered in physics, like the Navier-Stokes equations, in that they are much more dubious in nature. They are known as the equations of neural networks, or artificial neural networks, but that name is a misnomer; they were first invented back in the 1940s, when biologists barely knew anything about real neurons. At that time, microscopy was still in its infancy, and no one expected the equations to be truly faithful representations of real neurons; it was barely even known what was supposed to be represented.
Since then, neuroscientists have continued to view neural networks with skepticism. They see them as a shot in the dark at how groups of neurons function, a stab at a biologically meaningful representation. They have not generally been expected to be the bedrock of brain simulations. They do represent a situation a little akin to biological reality, but this representation is a supreme abstraction; it includes only a handful of aspects of the reality of real neurons. These include the fact that neurons carry signals; that these signals move through interconnections between them; that these connections can have different strengths; that some connections amplify signals, and others inhibit them; and so on. Although this might seem a fair few things to represent with an equation, the reality of a neuron is far more complicated.
Nonetheless, the reason why these models were invented was because they were seen, early on, to be valuable for certain purposes. Indeed, it was shown in the 1950s that they could be useful for processing certain kinds of information, like images, after that information was converted into the form of signals. This is not hard to do; for example, just convert each pixel of an image into a signal of zero strength, for a light pixel, and maximum strength, for a dark one. From there, those signals can be passed as input data into the neural network equations.
For many years and decades afterwards, neural networks lingered in a kind of scientific plateau territory. They were found to be useful for various purposes, but never fully adequate as either models of brain tissues, desired by neuroscientists, or as equations for processing information, desired by computer scientists and AI scientists.
But in the period from 2012-2014, both these camps suddenly made huge strides forward. In 2012, computer scientists discovered that neural networks could be made to solve some of the most challenging problems of human cognition, like recognizing unique objects regardless of their lighting, angle, or vantage point.4 In doing so, they established a major new correspondence between AI programs and the human mind, but one which was purely functional. This discovery came to be known as the ‘deep learning’ revolution, a terminology that I will return to.
Subsequent discoveries in biological sciences proved just as important. In 2014, DiCarlo and other neuroscientists built on the AI results, showing that AI programs and the brain, while both taking on these same impressive challenges, could also share strong correspondences in their internals. The signals in the models were strongly correlated with the real brain signals.
With these results, both neuroscientists and AI scientists together contributed to starting a kind of renaissance for neural networks in the great worldwide community of science. Both their efforts suggested that the programs most often conceived of as AI programs could actually have a lot in common with real brain tissues.
Over the next decade, the evidence for those commonalities would be strengthened in various ways that were sometimes startling. In particular, AI scientists and neuroscientists who studied language would go through a similar process of parallel discovery, where each discovered that the creations of the other could be surprisingly successful, either as models of brain regions, or as programs for solving cognitive challenges.
Nevertheless, not all biological scientists would be equally convinced about this progress. It is much more difficult to assess these AI programs as models of the brain, compared to assessing their pure brute force ability for solving problems. For one thing, the brain and mind are much more difficult to study and measure compared to computer programs. And, there was a wealth of understanding that had been gained, over many decades, that showed that there was still a gap between the models and the brain that could seem to yawn wide open.
1.6. Doubts and Criticisms
One of the biological scientists who has doubts about the validity of these new AI models is the psychologist Jeffrey Bowers, of the University of Bristol. I spoke to him in December 2022, when in the prior year he had become notorious for his criticisms. His manner was somewhat guarded, and I could see he possessed an independent streak; a stubborn desire to pursue the truth, over the trends—something common amongst all good scientists. Bowers had been studying human vision for decades, and had seen many different approaches come and go; undoubtedly, this personality trait of independence that he possessed stayed constant.
In a remarkable, December 2022 issue of the journal Behavioral and Brain Sciences5, he and several co-authors laid out the case for why modern AI models of the visual cortex were inadequate. He questioned the claims, which were becoming common, that they had become the leading models of human vision and the human visual cortex.
In page after page, section after section, he and his coauthors laid out the hard evidence for how the models failed to represent the reality of human vision. Some of the differences between the models and the mind could be downright drastic; like the fact that you could often fool a model from recognizing an object merely by altering a few tiny pixels in an image of it—a trivial problem for the mammalian mind to sort out. This came to be known as the problem of robustness (or often, adversarial robustness).
For Bowers, these AI models were not ‘good’ models of the visual cortex, although he acknowledged, perhaps a little grudgingly, some of their benefits. He would also scoff at the notion of calling them simulations of brain regions. He would probably feel even harsher when it would come to evaluating the realism of models of other brain regions, like the language network, or the auditory cortex, which have been created in the years since the initial 2012-2014 research landmarks. In other words, Bowers is was what you might call a simulation antagonist, even though he does not frame his criticisms of current AI models in the terminology of simulation.
However, the simulation interpretation undeniably provides an excellent way to encapsulate both his and others’ arguments. To think of these models as simulations means to endorse their realism, or the idea they are heading in that direction. On the other hand, to be opposed to calling them simulations is to question their realism; their status as leading models; or even their very foundations.
But the bottom line, regardless of how we frame the discussions, is that progress in the biological sciences is raising many questions and tensions amongst them. These are currently playing out in a great debate within the corridors, conferences, and public forums of neuroscience, and other subfields. It is fundamentally a question about the realism of what they are creating, when their current models are so much more realistic than any models that had ever previously been created, but still in many ways unrealistic.
In this debate, Bowers sticks out as someone unusually inclined to protestation. Most scientists have arguably gone in the opposite direction. For example, it is easier now to find neuroscientists talk openly of reverse engineering the brain, or crossing over into the territory of brain engineering. They talk openly of using AI programs as model organisms, the same way that biologists use mice as model organisms for humans, doing experiments on mice because they serve as models of the human. They broadly recognize how it is now possible to build AI programs that have a lot in common with real brains, or real brain regions.
But neither is Bowers alone in his doubts and criticisms. There are other biological scientists, sometimes of the highest pedigree and reputation, such as Noam Chomsky, arguably the greatest linguistic scientist in history, who ridicule current AI models of the mind and dismiss them.6 Some scientists have even suggested that they should be entirely abandoned. It is not hard to find someone from one side brusquely snapping at the other, if you know where to go looking.
In this book, I present both sides of this debate about whether we can see progress in modern neuroscience and AI as progress in creating realistic models of the brain, or brain simulations. But I don’t have any particular stance that I am defending. Rather, my point is that we can use the simulation interpretation, and the questions around it, to tell a fascinating story about where neuroscience and AI have taken us, and where they’re heading. The tensions in the interpretation of these new brain models do not have to be seen as something concerning or confusing, something for us to defer to the neuroscientists to sort out, in isolation. Rather, they give a truthful impression of how progress in science leads to challenging, but fully fair and legitimate questions.
1.7. Scrutinizing Industry
It might be nice if progress in neuroscience and AI was simple and settled. It would probably make it easier to fathom. But even though it isn’t, and neuroscientists aren’t sure what to make of their new creations, we must carefully consider them.
First, because if neuroscientists really are beginning to succeed in building brain region simulations, the evidence for which is highly suggestive, then the cosmic enigma of the mind has begun to be decoded. By building versions of the mind’s components, neuroscientists are making stunning progress in basic science that demands to be translated and communicated.
Second, regardless of how you interpret the advances of neuroscience, they can already offer hope, at least down the road, to those who suffer from brain injury or mental illness, or to those who have loved ones that are afflicted. Treating these problems depends on a fundamental understanding of what the brain is, which is exactly what these new brain models are providing; they show how the mind arises from its underlying implementation in biological tissues. Even now, these new AI models of the mind are already supporting the development of many practical applications of great medical importance, like prosthetic eyes, or head-mounted or intracranial devices that can decode a person’s brain signals into speech, or other modalities, something important for treating various disabilities, like locked-in syndrome.
But if biological scientists were merely creating brain simulations in the cloistered confines of academic and medical institutions, then the impacts of those creations would be insulated. We in the general public would have plenty of time to digest them, and we could trust in the medical profession’s established practices and norms of ethics and regulation.
However, as we have seen, what’s happening in AI is precisely the opposite. The sort of AI programs that neuroscientists are now making—in the form of putative brain region simulations—are now also commonly being made by AI scientists and engineers for largely industrial reasons. There is therefore a pressing need to examine what both of these fields now seem to be creating, and to examine them from a shared perspective, where the insights of the one field can be brought to bear on the other.
For example, it is already becoming common to hear talk of replacing creative workers, from industries like writing or software engineering, with modern AI programs, like ChatGPT, or its AI competitors from other companies, like Google, Baidu, and others. This would be a profoundly consequential economic and social decision, which in some cases, certain companies already seem to be taking.
The simulation interpretation provides an elegant but rigorous way of assessing the problems with this sort of decision. That is because neuroscientists who study human language have discovered that current language models are mainly only good models of a single, specific brain region, whereas humans are known to call on multiple distinctive brain regions for language processing. These include separate sub-processing modules for social and emotional reasoning, and for storing and retrieving existing knowledge. The lesson from neuroscience is therefore that to replace a human being with a current AI model like ChatGPT is to replace a human with a lobotomization.
The simulation interpretation thereby provides an essential means of circumscribing progress in AI, rather than hyping it. This is a little bit ironic, because one of the main reasons that neuroscientists may have shied away from the terminology of simulation is because they have been reluctant to claim that they have succeeded so far as to be creating brain simulations. But their insights into what the brain, which have been so hard won, may be essential for helping us understand AI’s limits.
At the same time, we must be careful not to overgeneralize between AI and neuroscience. The practitioners of these two fields tend to have very different purposes. For example, when either AI scientists or neuroscientists create AI programs, either for the purposes of making profitable software, or for the purposes of making models of the brain, respectively, they make many customizations. These customizations mean that programs like ChatGPT are very different from a neuroscientist’s simulation of a brain region.
Furthermore, AI scientists focus on raw performance in a way that is very different from neuroscientists. AI scientists seek quite generally to build programs that are as ‘intelligent’ as possible, whereas neuroscientists seek to build better models of brains, the intelligence of which is, by its very nature, limited. This has led AI scientists to make AI programs that are highly distorted in certain ways compared to neuroscientists’ AI programs.
Most especially, AI scientists have radically scaled up the number of artificial neurons in their AI programs, by comparison to neuroscientists. This had led their programs to become qualitatively and quantitatively different, even though they share many fundamentals in common. While an AI program made by an AI scientist might therefore be analogous to a brain region, that brain region might also be a little alien, in the same way an airplane is alien in comparison to a flying creature of nature.
The punchline of the simulation interpretation is therefore nuanced. It says that if you draw a Venn diagram with one circle representing AI programs, like ChatGPT, and one circle representing simulations of brain regions, made by neuroscientists, then these circles would have an intersection. This intersection is much greater than would historically have been believed, but it is still far from totality. And even with this nuance, the depth of commonality between AI and minds demands consideration.
1.8. The Worm Horizon
It is undeniable that the field of AI has made many major advances, which this book sheds light on. But it is right to view these with skepticism, especially when the science of AI is so mingled with marketing and corporate incentives. It has become common for the largest AI companies, like DeepMind (a subsidiary of Google), or OpenAI, to now talk in their mission statements of ‘solving intelligence.’ Some of their AI researchers even anticipate that the timeline to building broadly human-level AI, often known as artificial general intelligence (AGI), may be incipient.
This frenzied pace of activity is often seen in an optimistic light, where scientific advancement is equated with societal progress. But is the AI technology that is being developed actually helping society—or is it mainly being deployed for the financial benefit of the few, while actually harming the wider public?
This sort of question about the societal impacts of AI science is obviously of the greatest importance. It has become the focus of subfields known as AI ethics and AI safety. (Most often, these fields concentrate on AI models that are seen as information processing algorithms, rather than as models of cognitive functions or analogs of brain regions.) However, questions about scientific impacts are difficult to evaluate without possessing a firm understanding of the science—the technical problems that already have been solved, or haven’t.
In many ways, research in neuroscience now provides exactly the foundations needed for assessing AI’s advancement. We have already glimpsed how AI scientists—by adopting the methods of neural networks—adopted the same methods as neuroscientists. This was something of a historical coincidence; it could well have been the case that AI scientists adopted very different methods, which had nothing to do with human intelligence. But because they did, we can use neuroscience to help us gauge AI’s advances.
But there is another, even more fundamental reason why we must turn to neuroscience for insight into AI’s developments. That is because AI scientists will never be able to validate whether they are creating anything ‘intelligent’ without measuring them against a working definition of intelligence. And so far, in science, our only definitions of intelligence come from the study of living organisms. We do not know what intelligence is, as a purely abstract concept.
But if advances in AI must be measured against real biological organisms, then that suggests that AI is still a long ways off from achieving true, human-level intelligence. Because for all the advances that AI scientists have made, they have not yet come close to creating anything with comparable autonomy to even the simplest organisms—like the humble worm, C. elegans. To build a program with the intelligence of a worm—let alone a more complicated mind—appears to remain, at the moment, out of reach of AI scientists.
Indeed, when it comes to building whole entire minds, worm neuroscientists are arguably the ones laying the strongest roadmap. For them, building a simulation of the worm has become a common, unifying concept. Even though the OpenWorm project may have failed, today, many others are carrying the torch for worm simulation forward. They are established faculty at prestigious institutions, like MIT, the University of Vienna, Princeton University, and others. They undertake their efforts with the aid of significant funding from the general public. For them, the terminology of simulation is no longer taboo, a term to be avoided; it is an acknowledged endpoint.
Steven Flavell, an assistant professor at MIT, is one of the young leaders of this modern worm simulation movement. He has taken up the pioneering tools that have been created by other worm scientists in the last two decades, which allow them to perform feats that sound as science fiction as anything from the field of AI. With these tools, they can genetically engineer much of the worm’s nervous system, allowing them to make wholesale modifications to it. They routinely create worms whose neurons each fluoresce a different color, so they can distinguish each of them in action. They have designed spherical microscope slides to place their worms on, so that their views remain in focus even while the worms roam in free motion.
“You can cut right to the heart of the worm, and try to answer these questions that one day, maybe ten or twenty years from now, people will able to think about in a mouse, or a monkey, or a human,” Flavell told me in an interview. “It should be the first system where you have a full simulation.”
The work of neuroscientists like Flavell offers a window into the worm that is in fact, more like a grand vista, more like gazing into a grand canyon. Worm researchers have shown that some of its single neurons are, by themselves, akin to the complete sensory cortices of humans. For example, a single neuron of a worm might act like a little nose, all by itself, detecting certain scents (or chemicals) and reporting those back to the entire nervous system. On the other hand, groups of neurons still must work together to perform complex tasks like head movement, locomotion, or feeding. The worm possesses a majestic mind, even if it is simple, by comparison to our own one.
But although we have come to understand much about the worm, it is still offers a humbling lesson. We still do not understand the way that its mind functions. In his two decades of experience as a researcher, the best hypothesis that Flavell has so far come up with is that the mind of the worm is really controlled by its underlying chemical system, something entirely missing from modern AI programs. The mind of the worm is driven not by raw cognition, but by basic neurotransmitters like dopamine and serotonin, which also exist in humans, and are implicated in all our happiness and depression.
The powerful tech companies who are pushing AI forward often skip straight past these problems. Their tendency is to pursue advances that are expeditious, rather than positioned at the crux of the challenge. For example, in the last several years, they have focused on creating AI programs that can perform multiple sensory functions, like processing both text and images. Doing so has allowed them to make more mind-like creations.
But the lesson from neuroscience is that, despite the impressiveness of these constructions, we are still only beginning to understand how whole minds function. That can serve as an antidote, of sorts, to AI’s breathless advancement. There is still much unknown and unique about what makes us human, and it is likely to remain that way for some time.
In sum, it will be a long voyage to get to the end of the cosmos inside of us. I hope this book can help you understand what has made the journey so fascinating.
Sarma, Gopal P., et al. “OpenWorm: Overview and Recent Advances in Integrative Biological Simulation of Caenorhabditis Elegans.” Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 373, no. 1758, Oct. 2018, p. 20170382. DOI.org (Crossref), https://doi.org/10.1098/rstb.2017.0382.
Krizhevsky, Alex, et al. “ImageNet Classification with Deep Convolutional Neural Networks.” Advances in Neural Information Processing Systems, vol. 25, Curran Associates, Inc., 2012. Neural Information Processing Systems, https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html.