Ambient Intelligence Systems are complex, knowledge providing systems. The group of intended users typically comprises both various kinds of professionals and of laymen. In this chapter we address responsibility issues associated with the use and design of AmISs. We explain the connection between knowledge and responsibility, and use that connection to show that the usage of Ambient Intelligence Systems leads to moral responsibilities for all types of users (professionals and otherwise). It follows from the same reasoning scheme applied to the role of the AmIS developer, that she is responsible to design the system for responsible use. We give some initial criteria for such design for responsibility of AmISs.