Increasing understanding of how to categorize patient symptoms for efficient diagnosis has led to structured patient interviews and diagnostic flowcharts that can provide diagnostic accuracy and save valuable physician time. But the rigidity of predefined questions and controlled vocabulary for answers can leave patients feeling over-constrained, like the doctor (or computer system) is not really listening to them. In addition, not hearing the patient's own words can lead to the physician overlooking subtle details that are diagnostically relevant. How can we reconcile the need for patients to express themselves with the doctor's need to understand the patient's experience in medically appropriate terms?

We present I'm Listening, a system for automatically conducting patient pre-visit interviews. It does not replace a human doctor, but can be used before an office visit to elicit complaint details. This information can be used to triage care and prepare patients for visits with educational materials and appropriate tests, making better use of both doctor and patient time. It uses an on-screen avatar and natural language processing to (partially) understand the patient's response. Key is a Commonsense reasoning system that lets patients express themselves in unconstrained natural language, even using metaphor, and that maps the language to medically relevant categories. For example, if a patient describes his or her pain like, “someone sticking in a knife and then turning it”, the system could categorize it as sharp, intense, and localized.