The field of eXplainable Artificial Intelligence (xAI) is increasingly recognizing the need to personalize and interactively adapt the explanation to reflect better users’ explanation needs. While dialogue-based approaches to xAI have been proposed recently, the state-of-the-art in xAI is still characterized by what we call one-shot, non-personalized, and one-way explanations. In contrast, dialogue-based systems that can adapt explanations through interaction with a user promise to be superior to GUI-based or dashboard explanations as they offer a more intuitive way of requesting information. In general, while interactive xAI systems are often evaluated in terms of user satisfaction, there are limited studies that access user’s objective model understanding. This is in particular the case for dialogue-based xAI approaches. In this paper, we close this gap by carrying out controlled experiments within a dialogue framework in which we measure the understanding of users in three phases by asking them to simulate the predictions of the model they are learning about. By this, we can quantify the level of understanding of how the model works, comparing the state before and after the interaction. We further analyze the data to reveal patterns of how the interaction between groups with high vs. low understanding differs. Overall, our work thus contributes to our understanding of the effectiveness of xAI approaches.