Intelligent environments communicate with the user through user interfaces, which can be improved by means of adapting them to an individual user, based on an observation of the user-system interaction. The power of the adaptations depends on the expressiveness of the underlying data model that represents the interactive system, the user, and the user-system interaction. Ontologies present a powerful formalism for describing a certain domain. We present an adaptation framework that employs an ontology to create an explicit semantic knowledge representation, on top of which adaptations are implemented. These adaptations are either specified by a system designer or inferred automatically by an adaptation component. Special emphasis is put on the user-system interaction, which plays a crucial role in adaptive interactive systems. A prototype implementation demonstrates the feasibility of this approach.