In this paper we review our work on the acquisition of game-playing capabilities by a computer, when the only source of knowledge comes from extended self-play and sparsely dispersed human (expert) play. We summarily present experiments that show how a reinforcement learning backbone coupled with neural networks for approximation can indeed serve as a mechanism of the acquisition of game playing skill and we derive game interestingness measures that are inexpensive and straightforward to compute, yet also capture the relative quality of the game playing engine. We draw direct analogues to classical genetic algorithms and we stress that evolutionary development should be coupled with more traditional, expert-designed paths. That way the learning computer is exposed to tutorial games without having to revert to domain knowledge, thus facilitating the knowledge engineering life-cycle.