Ebook: Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms

The topic of this book – the creation of software programs displaying broad, deep, human-style general intelligence – is a grand and ambitious one. And yet it is far from a frivolous one: what the papers here illustrate is that it is a fit and proper subject for serious science and engineering exploration. No one has yet created a software program with human-style or (even roughly) human-level general intelligence – but we now have a sufficiently rich intellectual toolkit that it is possible to think about such a possibility in detail, and make serious attempts at design, analysis and engineering. This is the situation that led to the organization of the 2006 AGIRI (Artificial General Intelligence Research Institute) workshop; and to the decision to pull together a book from contributions by the speakers at the conference.

The themes of the book and the contents of the chapters are discussed in the Introduction by myself and Pei Wang; so in this Preface I will restrict myself to a few brief and general comments.

As it happens, this is the second edited volume concerned with Artificial General Intelligence (AGI) that I have co-edited. The first was entitled simply Artificial General Intelligence; it appeared in 2006 under the Springer imprimatur, but in fact most of the material in it was written in 2002 and 2003. It is interesting to compare the material contained in the present volume, which was written in 2006, with the material from the previous volume. What is striking in performing this comparison is the significant movement toward practical realization that has occurred in the intervening few years.

The previous volume contained some very nice mathematical theory (e.g. by Marcus Hutter and Juergen Schmidhuber) pertaining to AGI under assumptions of nearinfinite computational resources, some theory about the nature of intelligence as pertaining to AGI, and some descriptions of practical AGI projects at fairly early stages of development (including the NARS and Novamente systems developed by Pei Wang and myself respectively). The current volume, on the other hand, seems to represent significant progress. To take just a few examples: In the current volume, there is theoretical work (Eric Baum's and Moshe Looks' papers) that takes up Hutter's and Schmidhuber's emphasis on algorithmic information, and ties it in with practical suggestions regarding near-term AI design. My own Novamente system, which was described in fairly abstract terms in the earlier volume, is here represented by several papers by various authors reporting specific mathematical and experimental results, concretizing some (though by no means all, yet!) of the speculations made in the paper on Novamente in the previous volume. And, here we have a sufficient number of AGI design proposals, depicted in sufficient detail, that we have considered it worthwhile to include a chapter specifically comparing and contrasting four of the designs presented herein (Novamente, NARS, and the designs proposed by Stan Franklin and Alexei Samsonovich in their chapters).

In sum, what seems evident in comparing the prior volume with this one is that, while the end goal of the AGI research programme has not yet been achieved (and the proximity of achievement remains difficult to objectively predict), the field is gradually broadening its scope beyond mathematical and conceptual ideas, and becoming more of a practical pursuit.

And I am confident that if there is another edited volume in another 2 or 3 years time, the field will appear yet further dramatically advanced. The “AGI Winter” is thawing, and the AI field is now finally making sensible progress toward its original goal of creating truly thinking machines. The material presented here only scratches the surface of the AGI-related R&D work that is occurring around the world at this moment. But I am pleased to have had the chance to be involved in organizing and presenting at least a small percentage of the contemporary progress.

Finally, thanks must be extended to those who helped this volume, and the workshop that inspired it, to come into being. Bruce Klein deserves the lion's share of thanks, as the 2006 AGIRI Workshop would not have come into being without his extraordinary vision and dedication. Everyone who attended the workshop also deserves a piece of gratitude, and especially those who spoke or participated in panel discussions. Anya Kolonin did a fine job of reformatting the manuscript for publication. And finally I must extend heartfelt thanks to my co-editor Pei Wang for his part in helping to pull together this book, and the scientific program of the workshop that inspired it. In my work with him over the years I have found Pei to display a combination of good sense, insight and reliability that is distressingly rare in this world (populated as it is by mere humans … for now…).

This introductory chapter sets the stage for the book as a whole. First, the purpose of the conference that led to the book is reviewed. Then, the notion of “Artificial General Intelligence” (AGI) is clarified, including a brief survey of the past and present situation of the field, an analysis and refutation of some common objections and doubts regarding the AGI area of research, and a discussion of what needs to be addressed by the field as a whole in the near future. Finally, there is a summary of the contents of the other chapters in the book.

This chapter is a survey of a large number of informal definitions of “intelligence” that the authors have collected over the years. Naturally, compiling a complete list would be impossible as many definitions of intelligence are buried deep inside articles and books. Nevertheless, the 70 odd definitions presented here are, to the authors' knowledge, the largest and most well referenced collection there is.

During his talk at the 2006 AGI Workshop, Stan Franklin suggested that his LIDA architecture might fruitfully be considered not only as a specific AGI design, but also as a general framework within which to discuss and compare various AGI designs and approaches. With this in mind, following the workshop itself, Ben Goertzel formulated a list of simple questions intended to be pertinent to any AGI software design, mostly based on the conceptual framework presented in Stan Franklin's workshop presentation, with a couple additions and variations. Four individuals (Goertzel, Franklin, Pei Wang and Alexei Samsonovich) took up this offer, and their answers are reported here, without modification. The result is an interesting and novel perspective of a significant slice of contemporary AGI research.

Implementing and fleshing out a number of psychological and neuroscience theories of cognition, the LIDA conceptual model aims at being a cognitive “theory of everything.” With modules or processes for perception, working memory, episodic memories, “consciousness,” procedural memory, action selection, perceptual learning, episodic learning, deliberation, volition, and non-routine problem solving, the LIDA model is ideally suited to provide a working ontology that would allow for the discussion, design, and comparison of AGI systems. The LIDA architecture is based on the LIDA cognitive cycle, a sort of “cognitive atom.” The more elementary cognitive modules and processes play a role in each cognitive cycle. Higher-level processes are performed over multiple cycles. In addition to giving a quick overview of the LIDA conceptual model, and its underlying computational technology, we argue for the LIDA architecture's role as a foundational architecture for an AGI. Finally, lessons For AGI researchers drawn from the model and its architecture are discussed.

Humans can construct powerful mental programs for many domains never seen before. We address the questions of how this occurs, and how it could possibly be accomplished in software. Section one surveys a theory of natural understanding, as follows. One understands a domain when one has mental programs that can be executed to solve problems arising in the domain. Evolution created compact programs understanding domains posed by nature. According to an extrapolation of Occam's razor, a compact enough program solving enough problems drawn from a distribution can only be found if there is simple structure underlying the distribution and the program exploits this structure, in which case the program will generalize by solving most new problems drawn from the distribution. This picture has several important ramifications for attempts to develop Artificial General Intelligence (AGI), suggesting for example, that human intelligence is not in fact general, and that weak methods may not suffice to reproduce human abilities. Section 2 exemplifies this picture by discussing two particular thought processes, the mental program by which I solve levels in the game of Sokoban, and how I construct this mental program. A computer program under construction to implement my introspective picture of my mental program is based on a model of a particular mental module called Relevance Based Planning (RBP). Section 3 argues that programs to address new problems (such as my mental program to play Sokoban) can be constructed (both naturally and artificially) if and only if sufficient guidance is already present. It proposes a computational structure called a scaffold that guides rapid construction of understanding programs when confronted with new challenges.

NARS is an AGI project developed in the framework of reasoning system, and it adapts to its environment with insufficient knowledge and resources. The development of NARS takes an incremental approach, by extending the formal model stage by stage. The system, when finished, can be further augmented in several directions.

The goal of this chapter is to outline the attention machine computational framework designed to make a significant advance towards creating systems with human-level intelligence (HLI). This work is based on the hypotheses that: 1. most characteristics of human-level intelligence are exhibited by some existing algorithm, but that no single algorithm exhibits all of the characteristics and that 2. creating a system that does exhibit HLI requires adaptive hybrids of these algorithms. Attention machines enable algorithms to be executed as sequences of attention fixations that are executed using the same set of common functions and thus can integrate algorithms from many different subfields of artificial intelligence. These hybrids enable the strengths of each algorithm to compensate for the weaknesses of others so that the total system exhibits more intelligence than had previously been possible.

The notion of a human value system can be quantified as a cognitive map, the dimensions of which capture the semantics of concepts and the associated values. This can be done, if one knows (i) how to define the dimensions of the map, and (ii) how to allocate concepts in those dimensions. Regarding the first question, experimental studies with linguistic material using psychometrics have revealed that valence, arousal and dominance are primary dimensions characterizing human values. The same or similar dimensions are used in popular models of emotions and affects. In these studies, the choice of principal dimensions, as well as scoring concepts, was based on subjective reports or psycho-physiological measurements. Can a cognitive map of human values be constructed without testing human subjects? Here we show that the answer is positive, using generally available dictionaries of synonyms and antonyms. By applying a simple statistical-mechanic model to English and French dictionaries, we constructed multidimensional cognitive maps that capture the semantics of words. We calculated the principal dimensions of the resultant maps and found their semantics consistent across two languages as well as with previously known main cognitive dimensions. These results suggest that the linguistically derived cognitive map of the human value system is language-invariant and, being closely related to psychometrically derived maps, is likely to reflect fundamental aspects of the human mind.

A program evolution component is proposed for integrative artificial general intelligence. The system's deployment is intended to be comparable, on Marr's level of computational theory, to evolutionary mechanisms in human thought. The challenges of program evolution are described, along with the requirements for a program evolution system to be competent – solving hard problems quickly, accurately, and reliably. Meta-optimizing semantic evolutionary search (MOSES) is proposed to fulfill these requirements.

This chapter shows how a “Celoxica” electronic board (containing a Xilinx Virtex II FPGA chip) can be used to accelerate the evolution of neural network modules that are to be evolved quickly enough, so that building artificial brains that consist of 10,000s of interconnected modules, can be made practical. We hope that this work will prove to be an important stepping stone towards making the new field of brain building both practical and cheap enough for many research groups to start building their own artificial brains.

The main finding of complex systems research is that there can be a disconnect between the local behavior of the interacting elements of a complex system and regularities that are observed in the global behavior of that system, making it virtually impossible to derive the global behavior from the local rules. It is arguable that intelligent systems must involve some amount of complexity, and so the global behavior of AI systems would therefore not be expected to have an analytic relation to their constituent mechanisms. This has serious implications for the methodology of AI. This paper suggests that AI researchers move toward a more empirical research paradigm, referred to as “theoretical psychology,” in which systematic experimentation is used to discover how the putative local mechanisms of intelligence relate to their global performance. There are reasons to expect that this new approach may allow AI to escape from a trap that has dogged it for much of its history: on the few previous occasions that something similar has been tried, the results were both impressive and quick to arrive.

A novel theory of stages in cognitive development is presented, loosely corresponding to Piagetan theory but specifically oriented toward AI systems centered on uncertain inference components. Four stages are articulated (infantile, concrete, formal and reflexive), and are characterized both in terms of external cognitive achievements (a la Piaget) and in terms of internal inference control dynamics. The theory is illustrated via the analysis of specific problem solving tasks corresponding to the different stages. The Novamente AI Engine, with its Probabilistic Logic Networks uncertain inference component and its embodiment n the AGI-SIM simulation world, is used as an example throughout.

The creation of robust mechanisms for uncertain inference is central to the development of Artificial General Intelligence systems. While probability theory provides a principled foundation for uncertain inference, the mathematics of probability theory has not yet been developed to the point where it is possible to handle every aspect of the uncertain inference process in practical situations using rigorous probabilistic calculations. Due to the need to operate within realistic computational resources, probability theory presently requires augmentation with heuristics in order to be pragmatic for general intelligence (as well as for other purposes such as large-scale data analysis).

The authors have been involved with the creation of a novel, general framework for pragmatic probabilistic inference in an AGI context, called Probabilistic Logic Networks (PLN). PLN integrates probability theory with a variety of heuristic inference mechanisms; it encompasses a rich set of first-order and higher-order inference rules, and it is highly flexible and adaptive, and easily configurable. This paper describes a single, critical aspect of the PLN framework, which has to with the quantification of uncertainty. In short, it addresses the question: What should an uncertain truth value be, so that a general intelligence may use it for pragmatic reasoning?

We propose a new approach to quantifying uncertainty via a hybridization of Walley's theory of imprecise probabilities and Bayesian credible intervals. This “indefinite probability” approach provides a general method for calculating the “weight-of-evidence” underlying the conclusions of uncertain inferences. Moreover, both Walley's imprecise beta-binomial model and standard Bayesian inference can be viewed mathematically as special cases of the more general indefinite probability model. Via exemplifying the use of indefinite probabilities in a variety of PLN inference rules (including exact and heuristic ones), we argue that this mode of quantifying uncertainty may be adequate to serve as an ingredient of powerful artificial general intelligence.

The Novamente Cognition Engine (NCE) architecture for Artificial General Intelligence is briefly reviewed, with a focus on exploring how the various cognitive processes involved in the architecture are intended to cooperate in carrying out moderately complex tasks involving controlling an agent embodied in the AGI-Sim 3D simulation world. A handful of previous conference papers have reviewed the overall architecture of the NCE, and discussed some accomplishments of the current, as yet incomplete version of the system; this paper is more speculative and focuses on the intended behaviors of the NCE once the implementation of all its major cognitive processes is complete. The “iterated Easter Egg Hunt” scenario is introduced and used as a running example throughout, due to its combination of perceptual, physical-action, social and self-modeling aspects. To aid in explaining the intended behavior of the NCE, a systematic typology of NCE cognitive processes is introduced. Cognitive processes are typologized as global, operational or focused; and, the focused processes are more specifically categorized as either forward-synthesis or backward-synthesis processes. The typical dynamics of focused cognition is then modeled as an ongoing oscillation between forward and backward synthesis processes, with critical emergent structures such as self and consciousness arising as attractors of this oscillatory dynamic. The emergence of models of self and others from this oscillatory dynamic is reviewed, along with other aspects of cognitive-process integration in the NCE, in the context of the iterated Easter Egg Hunt scenario.

Logic-based AI is often thought of as being restricted to highly abstract domains such as theorem-proving and linguistic semantics. In the Novamente AGI architecture, however, probabilistic logic is used for a wider variety of purposes, including simple reinforcement learning of infantile behaviors, which are primarily concerned with perception and action rather than abstract cognition. This paper reports some simple experiments designed to validate the viability of this approach, via using the PLN probabilistic logic framework, implemented within the Novamente AGI architecture, to carry out reinforcement learning of simple embodied behaviors in a 3D simulation world (AGISim). The specific experiment focused upon involves teaching Novamente to play the game of “fetch” using reinforcement learning based on repeated partial rewards. Novamente is an integrative AGI architecture involving considerably more than just PLN; however, in this “fetch” experiment, the only cognitive process PLN is coupled with is simple perceptual pattern mining; other Novamente cognitive processes such as evolutionary learning and economic attention allocation are not utilized, so as to allow the study and demonstration of the power of PLN on its own.

At the end of the 2006 AGI Workshop, a number of presenters were invited to participate in a panel discussion on the theme “How Do We More Greatly Ensure Responsible AGI?” This chapter presents a record of that dialogue, including audience participation, lightly edited by the presenters themselves for greater readability.

At the end of the 2006 AGI Workshop, a number of presenters were invited to participate in a panel discussion on the theme “What Are the Bottlenecks, and How Soon to AGI?” This chapter presents a record of that dialogue, including audience participation, lightly edited by the presenters themselves for greater readability.