AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence — part 2/8

Cleaned-text code points [15874, 31004). Bibliography omitted here; source.md includes it.

AI-GAs may prove to be the fastest path to general AI. However, even if they are not, they are worth pursuing anyway. It is intrinsically scientifically worthwhile to attempt to answer the question of how to create a set of simple conditions and a simple algorithm that can bootstrap itself from simplicity to produce general intelligence. Just such an event happened on Earth, where the extremely simple algorithm of Darwinian evolution ultimately produced human intelligence. Thus, one reason that creating AI-GAs is beneficial is that doing so would shed light on the origins of our own intelligence.

AI-GAs would also teach us about the origins of intelligence generally, including elsewhere in the universe. They could, for example, teach us which conditions are necessary, sufficient, and catalyzing for intelligence to arise. They could inform us as to how likely intelligence is to emerge when the sufficient conditions are present, and how often general intelligence emerges after narrow intelligence does.

Presumably different instantiations of AI-GAs (either different runs of the same AI-GA or different types of AI-GAs) would lead to different kinds of intelligence, including different, alien cultures. AI-GAs would likely produce a much wider diversity of intelligent beings than the manual path to creating AI, because the manual path would be limited by our imagination, scientific understanding, and creativity. We could even create AI-GAs that specifically attempt to create different types of general AI than have already been produced. AI-GAs would thus better allow us to study and understand the space of possible intelligences, shedding light on the different ways intelligent life might think about the world.

The creation of AI-GAs would enable us to perform the ultimate form of cultural travel, allowing us to interact with, and learn from, wildly different intelligent civilizations. Having a diversity of minds could also catalyze even more scientific discovery than the production of a single general AI would. We would also want to reverse engineer each of these minds for the same reasons we study neuroscience in animals, including humans: because we are curious and want to understand intelligence. We and others have been conducting such ‘AI Neuroscience’ for years now [ 120 , 122 , 124 , 121 , 200 , 102 , 199 , 180 , 107 , 201 , 100 , 54 ] , but the work is just beginning. John Maynard Smith wrote the following about evolutionary systems: “So far, we have been able to study only one evolving system, and we cannot wait for interstellar flight to provide us with a second. If we want to discover generalizations about evolving systems, we will have to look at artificial ones.” [ 108 ] The same is true regarding intelligence.

Additionally, explained in Section 2.3 , the manner in which AI-GA is likely to produce intelligence will involve the creation of a wide diversity of learning environments, which could include novel virtual worlds, artifacts, riddles, puzzles, and other challenges that themselves will likely be intrinsically interesting and valuable. Many of those benefits will begin to accrue in the short-term with AI-GA research, so AI-GAs will provide short-term value even if the long-term goals remain elusive. AI-GAs are also worthwhile to pursue because they inform our attempts to create open-ended algorithms (those that endlessly innovate) [ 172 ] . As described in Section 1.2 , we should also invest in AI-GA research because it might prove to be the fastest way to produce general AI.

For all these reasons, I argue that the creation of AI-GAs should be considered its own independent scientific grand challenge.

### 1.5 Both the manual and AI-GA paths are worth pursuing, irrespective of which is more likely to be the fastest path to general AI

Both the manual and AI-GA paths to creating general AI should be pursued, regardless of which one is considered the most likely to succeed first. The clearest argument for that is that even if one succeeds, we would still want to pursue the other. To see why, consider the possibility of each being the first to produce general AI.

If the manual path wins, we would still be interested in creating AI-GAs for all the reasons just provided in Section 1.4 . If the AI-GA path wins, we would still be interested in creating general AI via the manual path. That is because it is likely that an AI-GA would produce a complex machine whose inner workings we do not understand. As Richard Feynman said, “What I cannot create, I do not understand.” We understand by building, and one great way to understand intelligence is to be able to take it apart, rewire components, and learn how to put it together piece by piece. AI-GAs could actually catalyze the manual path because it would produce a large diversity of general AIs, perhaps making it easy for us to recognize what is necessary, variable, and incidental in the creation of intelligent machines. Additionally, each path would catalyze the other because the general AI produced by either could be used to accelerate scientific progress in the other path.

For all of these reasons, I consider the manual and AI-GA paths to creating general AI to be separate scientific grand challenges each worthy of substantial scientific investment. However, because the current machine learning community is mostly committed to the manual path, I advocate that we should shift investment to the AI-GA path to pursue both promising paths to general AI.

### 1.6 Which path is more likely to produce general AI first?

There are good reasons why either of the two paths might be the fastest way to produce general AI. However, on balance, after weighing all of the arguments presented in this manuscript, I believe it more likely that the AI-GA path will produce general AI first. That said, I have high uncertainty about that predication and think that either could get there first.

There are a few main reasons why I think the AI-GA path is more likely. Initially, it scales well with the exponential compute we can expect in the years and decades to come. While AI-GAs will require extraordinary amounts of computation by today’s standards, they have the nice property that they will be able to consume that compute easily once it is available. As Richard Sutton has recently pointed out, history has shown that the algorithms that tend to win over the long haul are simple ones that can take advantage of massive amounts of computing [ 178 ] . Additionally, as noted above, history teaches us that learned pipelines eventually surpass ones that are entirely or partially hand-created. Further, the AI-GA path enables small teams to begin working on algorithms that might produce general AI right now. In contrast, as noted above, the manual path eventually requires a switch to the Herculean Phase 2 task of putting together all of the building blocks of intelligence. If such a combination requires a very large team dedicated for years (something akin to the Apollo program), then major changes in the social organization of scientific effort must take place before work can begin on anything that has a chance at producing an intelligent machine via the manual path. That work must also be undertaken after, or concurrently with, discovering all of the building blocks of intelligence (Phase 1 of the manual path). Additional reasons why the AI-GA path might win are given in Sections 1.1 and 1.2 . Overall, it is likely that in the near to middle term, the manual path will produce higher-performing AI systems that are of more use to society, just as HOG and SIFT produced better computer vision systems in the pre-2012 era. However, just as with computer vision, it is likely that fully learned systems will eventually surpass and then rapidly far exceed the capabilities produced by the manual path.

## 2 Research into the Three Pillars

Researching how to create an AI-GA is a very different research agenda from the manual path. Instead of attempting to identify and create the individual building blocks of an intelligent machine (e.g. better memory modules, hierarchical controllers, optimizers, and transformers), AI-GAs attempt to learn all of these types of building blocks. That does not mean it is not without its own challenging research problems. It is just that the research problems are different and require focusing on different questions.

This section describes each of AI-GA’s Three Pillars, including prior work in them and a sketch of what future research in them might look like.

### 2.1 The First Pillar: Meta-learning Architectures

The First Pillar is the most studied and well-known, and for that reason I will spend little time surveying it. The architecture of a neural network plays a key role in both the function a trained neural network can perform and how easy it is to train that network [ 61 , 68 , 181 , 52 ] . As mentioned above, the majority of work into training neural networks with backpropagation has involved humans hand-designing architectures, including innovations such as convolution [ 91 ] , LSTMs or other gated recurrent units [ 65 , 23 , 24 ] , highway networks and residual networks [ 61 , 164 ] , etc. However, there is also a long history in AI of creating algorithms to search for high-performing neural network architectures [ 90 , 169 , 55 , 198 , 59 ] . Starting recently, such architecture search algorithms are being tested with large-scale compute for deep neural network architectures, where they are sometimes finding better architectures than those that humans have designed. For example, automatically discovered architectures improved performance even on the classic benchmark of CIFAR and the challenging benchmark of ImageNET [ 81 , 140 , 137 , 114 , 203 ] , both of which had already been highly optimized by armies of human scientists. We can thus reasonably assume the trend will continue and that searching for architectures will replace the hand-designing of architectures in the years and decades to come. Architecture search is expensive, but so was backpropagation in deep neural networks in the year 2010. Eventually the computation will exist to make it routine and likely the best option. Additionally, there are techniques that can speed it up, such as surrogate models [ 103 , 84 , 9 , 137 ] or Generative Teaching Networks [ 129 ] , and new techniques will be invented to make it faster still.

It is clear that animal brains, including our own, are not large, homogenous, fully connected neural networks. Instead, their architectures consist of many heterogenous modules wired together in a particular way, with the entire design carefully sculpted by evolution [ 175 ] . It is daunting to consider creating such complex architectures by hand. With exponentially more compute, we instead can search for them with an outer loop learning algorithm that produces powerful, sample-efficient architectures primed for learning. Since the outer-loop algorithm produces architectures that are better at learning, people often call architecture search algorithms ‘meta-learning algorithms’, because they learn how to produce better learners. Another notion of meta-learning (learning better learning algorithms) is described in the next section.

Much research needs to be done to enable an effective architecture search algorithm. That includes research into encodings (aka representations) [ 168 , 171 , 170 , 203 , 55 ] , search operators, and the production of architectures that are regular [ 171 , 27 , 69 ] , modular [ 28 , 69 ] , and hierarchical [ 109 ] . There is the chance for such architecture search algorithms to discover architectural innovations like those colored black in Table 1 . Research in this space is taking off at the moment and will be useful whether the search is for a domain-specific neural network solution or for producing an AI-GA.

### 2.2 The Second Pillar: Meta-learning the learning algorithms

Because history teaches us we should learn as much as possible, that lesson should apply to the learning algorithms themselves. Doing so could improve the ceiling for what neural networks are able to learn about each task, their sample efficiency, their ability to generalize, and their ability to continuously learn many tasks. Work in this field is often called simply ‘meta-learning’ and dates back decades [ 66 , 31 , 161 , 148 , 40 , 187 , 45 , 191 , 37 , 152 , 5 , 136 , 101 ] . The idea is to ‘learn to learn’ such that an agent gets better at learning a new task sampled from a distribution of tasks over the course of training. Instead of learning a specific task, the idea is to get better at learning on any new task by becoming a better learner. There has been a recent surge of interest in this area, with work falling into roughly two families.

The first family, made famous by the Model-Agnostic Meta Learning (MAML) algorithm [ 45 ] involves pairing a learner (a neural network with a set of weights θ \theta ) with a predefined, fixed learning algorithm—such as stochastic gradient descent (SGD)—to solve a task selected from a distribution of tasks as fast as possible. The job of the learning algorithm is to produce an initial set of weights that can rapidly learn any task from the distribution. To do so, MAML differentiates through the learning process to find a set of initial weights that are quickly able to learn a new task when taking gradient steps via SGD in that new task. It has been shown that theoretically this paradigm can encode any learning algorithm [ 44 ] .

The second family involves training a recurrent neural network (RNN) in a context that requires learning and relies on the activations with the neural network to implement a learning algorithm. Because an RNN is a Turing-complete universal computer [ 155 ] , it too can encode any possible learning algorithm. The weights of the RNN are trained via an outer loop meta-learning algorithm. This approach has been well-studied in a supervised learning context [ 66 , 31 ] . As applied to reinforcement learning, it was concurrently introduced by two papers [ 191 , 37 ] . It is intuitively appealing because it resembles what happened on Earth, where an outer loop optimization algorithm (evolution) created animal brains, which are effective, sample-efficient learners. Note, however, that evolution is not required: many algorithms can be used for the outer-loop learning algorithm, such as policy gradients [ 191 , 37 ] . This type of meta-learning is also appealing because it could learn a better inner-loop learning algorithm than any currently available machine learning algorithms. On simple reinforcement learning tasks it has been demonstrated that this approach can learn to explore, exploit, and balance between the two [ 191 , 37 ] . It can also seemingly build an internal model of the world and plan within that model to behave like model-based RL methods instead of model-free RL methods, or at least the networks it produces pass (admittedly imperfect [ 3 ] ) tests designed to tell model-based methods from model-free methods [ 191 ] .