While adaptive systems are currently generally judged by their degree of intelligence (in terms of their ability to discover how to achieve goals), the critical measurement for the future will be where they fall on the spectrum of self. Once machines and software are able to strongly modify and improve themselves, the concepts of the self and agency will be far more important, determining not only what a particular system will eventually be capable of but how it will actually act. Unfortunately, so little attention has been paid to this fact that most people still expect that the basic cognitive architecture of a passive “Oracle” in terms of consciousness, self, and “free will” will be little different from that of an active explorer/experimenter with assigned goals to accomplish. We will outline the various assumptions and trade-offs inherent in each of these concepts and the expected characteristics of each – which not only apply to machine intelligence but humans and collective entities like governments and corporations as well.