In this paper, we propose an integrated structure of intelligent techniques to enhance easy-controllability of service robots for the elderly and people with disabilities in a smart home environment. Intelligent techniques have been implemented in human-robot interfaces according to the 3 categories of intelligence, toward enhancing controllability. First, a gesture-recognition system has been implemented toward a more intuitive/natural human-robot interface. Second, a task planning system plays a role to simplify possible commands by the user to control various kinds of devices/robotic agents, for given well-structured tasks. Third, we applied a life-long learning system for intention reading of the user to provide service tasks automatically in advance of his/her explicit command instruction.