In this chapter we discuss the system requirements and components of an adaptive smart-home service system. To achieve adaptivity in providing services to the user, the system needs to 1) sense the activity and state of the user and 2) customize service to the user's profile. To achieve this, three functional parts are developed and described. In the first part, we present behavior analysis of the user in a home environment based on multi-camera vision processing. In the second part, the concept of user profile is introduced and hierarchical reinforcement learning is employed as a technique to learn the user profile dynamically. The third part of the chapter discusses how to employ the user profile to control services to maximize user comfort and utility. Future work is discussed in conclusion.