In this paper, a speech-interfaced system for fostering group conversations is presented. The system captures conversation keywords and shows visual stimuli on a tabletop display. A stimulus can be a feedback to the current conversation or a cue to discuss new topics. This work describes the overall system architecture and highlights details about the design choices of the overall system, with a particular focus on the real-time implementation issues. A suitable speech enhancement front-end and a keyword spotter have been integrated on a common software platform for real-time audio processing, namely Nu-Tech, resulting in a helpful and flexible architecture for real-world applications in group conversation modeling scenarios. Such system characteristics, jointly with some experimental results obtained from simulations on recorded speech data, seem to confirm the efficacy of the approach motivating the development of further features and the experimentation in new scenarios.