AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence — part 6/8

Cleaned-text code points [76186, 91484). Bibliography omitted here; source.md includes it.

That said, the AI-GA approach is not free of building blocks. It does not start from scratch, trying to create intelligence from nothing but the laws of physics and a soup of atoms. It might, for example, start with the assumption that we want neural networks with regular and neuromodulatory neurons, niches and explicit goal-switching, and populations of agents within each environmental niche. Research on AI-GAs will involve identifying the minimum number of sufficient and catalyzing building blocks to create an AI-GA. But the set of building blocks AI-GA researchers will look for will be very different from those for the manual engineering approach. For example, AI-GA will likely attempt to automatically learn everything in Table 1 . Rather than discovering those building blocks manually, AI-GA researchers will try to figure out what are the building blocks that enable us to abstract open-ended complexity explosions in a computationally tractable way. I hypothesize that fewer building blocks need to be identified to produce an AI-GA than to produce intelligence via the manual AI approach. For that reason, I further hypothesize that identifying them and how to combine them will be easier, and thus that AI-GA is more likely to produce general AI faster. That said, an open question is how much compute is required to make an AI-GA. If we magically had vastly more compute available starting tomorrow, my confidence in AI-GAs producing general AI before the manual path would be greatly increased. What is less clear is whether AI-GAs can beat the manual path given that the computation AI-GAs need is not yet available. The lack of computation gives the manual path an advantage since it is more compute-efficient. On the other hand, as Rich Sutton has argued, history often favors algorithms that better take advantage of the exponential increases in computation the future provides [ 178 ] .

It may turn out that the manual path can succeed without having to identify hundreds of building blocks. It might turn out instead that only a few are required. At various points in the history of physics many pieces of theoretical machinery were thought to be required, only later to be unified into smaller, simpler, more elegant frameworks. Such unification could also occur within the manual AI path. But at some point if everything is being learned from a few basic principles, it seems more like an AI-GA approach instead of a manual identify-then-combine engineering approach. In other words, the AI-GA approach is the quest for a set of simple building blocks that can be combined to produce general AI, so true unification would validate it.

One interesting way to catalyze the Third Pillar is to harness data from the natural world when creating effective learning environments. This idea could accelerate progress in both the target-task and/or open-ended versions of the Third Pillar. For example, the environment generator could be given access to the Internet. It could then learn to generate tasks that involve classifying real images, imitating the skills animals and/or humans perform (e.g. in online video repositories such as Youtube), solving problems in existing textbooks, or solving existing machine learning benchmarks in language, logic, reinforcement learning, etc. There is a long history of fruitful research in imitation learning and learning via observation that demonstrates the benefits of exploiting such data [ 39 , 14 , 165 , 7 , 145 , 38 , 185 , 132 , 119 , 1 ] . AI-GAs too could benefit from this treasure trove of information. Incorporating such tasks might also have the benefit of making the AI that results more capable at solving problems in our world and better able to communicate with and understand us (and vice versa).

There is a third path to general AI. I call it the “mimic path.” It involves neuroscientists studying animal brains, especially the human brain, in an attempt to reverse engineer how intelligence works in animals and rebuild it computationally. In contrast to abstracting the general principles found via neuroscience and building them in any workable instantiation (which is the purview of the manual path), the mimic path attempts to recreate animal brains in as much faithful detail as possible. The most famous example of this approach is the work of Henry Markram and his Blue Brain and European Human Brain projects. This path is also independently worthwhile, and should be pursued irrespective of whether the other two paths have already succeed. That is because if the manual or AI-GA paths build general AI that does not resemble human brains, we would still be interested in specifically how human intelligence works. If the mimic path wins, we would still be interested in the other two paths for the reasons outlined earlier. The mimic path is unlikely to be the fastest path to producing general AI because it does not seek to benefit from the efficiencies of abstraction. For example, if an entire neocortical column could be functionally replaced by a multi-layer recurrent neural network with skip connections, the spirit of the mimic path would be to eschew that option in favor of faithfully replicating the actual human neocortical column with all of its expensive-to-simulate complexity. Additionally, the mimic path is slowed by the difficulty of creating the technologies required to identify what is happening inside functioning natural brains. Overall, I do not view the mimic path it as one of the major paths to producing general AI because the goal of those committed to it is not solely to produce general AI. I thus only mention it this late in this essay, and still consider the manual and AI-GA paths as the two main paths.

For the most part, this article has assumed that neural networks have a large part to play in the creation of AI-GAs. That reflects my optimism regarding deep neural networks, as well as my experience and passions. That said, it is of course possible that other techniques may produce general AI. One could substitute other techniques into the pillars of the AI-GA framework. For example, one could replace Pillar One and Pillar Two with Solomonoff induction via AIXI [ 73 , 72 ] or similar ideas. To do so, major advances would be required to make these approaches computationally tractable, as with DNNs. However, one would still need work on AI-GA’s Third Pillar because Solomonoff induction and AIXI take as a starting assumption a single environment to be solved. That highlights again that generating effective learning environments is the newest, least-explored area of research of the three pillars, even in far disparate areas of AI research. Advances in it may thus benefit many different subfields of AI.

I also want to emphasize that AI-GAs are not an “evolutionary approach,” despite people gravitating towards calling it that despite my saying otherwise. There are a few reasons to avoid that terminology. A first reason is that many methods could serve as the outer-loop optimizer. For example, one could use gradient-based meta-learning via meta-gradients [ 129 , 45 , 106 ] or policy gradients [ 191 , 37 ] . Additionally, one could potentially use Bayesian Optimization as the outer-loop search algorithm, although innovations would be needed to help them search in high-dimensional search landscapes. Of course, evolutionary algorithms are also an option. There are pros and cons to using evolutionary methods [ 176 , 96 , 146 , 195 , 167 ] , and benefits to hybrid approaches that combine evolutionary concepts (searching in the parameter space) with policy-gradient concepts (searching in the action space) [ 131 , 46 ] . Because they are just one of many choices, calling AI-GAs an evolutionary approach is inaccurate. A second reason to avoid calling this an evolutionary approach is that many people in the machine learning community seem to have concluded that evolutionary methods are not worthwhile and not worth considering. There is thus a risk that if the AI-GA idea is associated with evolution it will not be evaluated with a clear, objective, open-mind, but instead will be written off for reasons that do not relate to its merits.

Of course, in practice the three different paths will not exist isolated from each other. The manual, mimic, and AI-GA paths will all inform each other as discoveries in one catalyze and inspire work in the other two. Additionally, people will pursue hybrid approaches (e.g. the vision outlined in Botvinick et al. [16] ) and it will be difficult in many cases to tell which path a particular group or project belongs to. That said, it is instructive to give these different approaches their own names and talk about them separately despite the inevitable cross pollination and hybridization that will occur.

AI-GAs are a bet on learning in simulated worlds. The hypothesis is that we could create general AI entirely in such virtual worlds (although they may have to be complex worlds). However, once general AI is produced, that AI would be a sample-efficient learner that could efficiently learn in our world. For example, such an agent could be transferred to a robot that could learn to maneuver in our world, including helping and otherwise interacting with humans. Thus, rather than specifically designing techniques to transfer from simulation to reality, the approach would not require such adaptation because the general AI would perform that efficiently itself.

Many researchers will enjoy research into creating AI-GAs. There are many advantages to AI-GA research versus the manual path. Initially, there are currently tens of thousands of researchers pursuing the manual path. There are very few scientists currently pursuing AI-GAs. For many, that will make AI-GA research more exciting. It also decreases the risk of getting scooped. Additionally, to pursue the manual path, one might feel (as I have historically) the need to stay abreast of developments on every building block one considers potentially important to creating a large, complicated general AI machine. One might thus feel a desire to read all new, important papers on each of the building blocks in Table 1 , as well as building blocks it does not list and new ones as they are discovered. That is of course impossible given the number of such papers published each year. Because AI-GAs have fewer researchers and, if my hypothesis is correct, fewer building blocks required to make them, staying abreast of the AI-GA literature should prove more manageable. Additionally, if the large-scale trend in machine learning continues, wherein hand-coded components are replaced by learning, working on AI-GAs is a way to future-proof one’s research. To put it brashly, knowing what you know now, if you could go back 15 years ago, would you want to dedicate your career to improving HOG and SIFT? Or would you rather invest in the learning-based approaches that ultimately would render HOG and SIFT unnecessary, and prove to be far more general and powerful?

As AI-GA research advances, it will likely generate many innovations that can be ported to the manual path. It will also create techniques and narrow AIs of economic importance that will benefit society long before the ambitious goal of producing general AI is accomplished. For example, creating algorithms that automatically search for architectures, learning algorithms, and training environments that solve tasks will be greatly useful, even for tasks far less ambitious than creating general AI. Thus, even if creating an AI-GA ultimately proves impossible, research into it will be valuable.

There has been a long debate in the artificial intelligence and machine learning community as to where we should be on the spectrum between learning everything from scratch and hand-designing everything. Some argue that we need to inject lots of human priors into a system in order for it to be sample-efficient and/or generalize well. Others think we should learn as much as possible from data, eschewing human priors because learned solutions are ultimately better once there is sufficient compute and data. Where does the AI-GA paradigm fit in this debate? First, where we should be on the spectrum of course depends on our goals and how much time we have to achieve them. If we already know how to build a solution that performs well enough and performance is all we care about, we should clearly do that. Additionally, if we want to build a solution on a short timeframe, it is likely that the fastest path to a decent solution will involve lots of human prior information (unless we already know how to create learning-based solutions, as is the case in the many areas where deep learning is currently the dominant technique). However, for more ambitious goals over longer time horizons, it is often the case that betting on a learning-based solution is a good strategy.

However, a commitment to learning is not a commitment to sample inefficient learners that only perform well with extreme levels of data and that generalize poorly. The AI-GA philosophy is that via a compute-intensive, sample-inefficient outer loop optimization process we can produce learning agents that are extremely sample efficient and that generalize well. Just as evolution (a slow, compute-intensive, sample-inefficient process) produced human intelligence, as AI-GAs advance they will produce individual learners that increasingly approach human learners in sample efficiency and generalization abilities. One might argue that means that the system itself is not sample efficient, because it requires so much data. That is true in some sense, but not true in other important ways. One important meaning of sample efficiency is how many samples are needed on a new problem . For example, a new disease might appear on Earth and we may want doctors or AI to be able to identify it and make predictions about it from very few labeled examples. If an AI-GA produces anything akin to human-level intelligence, the learner produced would be able, as humans are, to be sample efficient with respect to this new problem. The idea behind AI-GAs is that as compute becomes cheaper and our ability to generate sufficiently complex learning environments grows, we can afford to be sample inefficient in the outer loop in the service of producing a learner that is at or beyond human intelligence in being sample efficient when deployed on problems we care about. Putting aside its computational cost, AI-GAs thus in some sense represent the best of both ends of the spectrum: it can learn from data and not be constrained by human priors, but can produce something that, like humans themselves, contain powerful priors and ways to efficiently update them to rapidly solve problems.

## 4 Safety and ethical considerations

Any discussion of producing general AI raises the ethical question of whether we should be pursuing this goal. The question of whether and why we should create general AI is a complicated one and is the focus of many articles [ 143 , 41 , 4 , 18 , 15 ] . I will not delve into that issue here as it is better served when it is the sole issue of focus. However, the AI-GA path introduces its own unique set of ethical issues that I do want to mention here.