Internationally, major efforts are underway to improve medication safety and reduce medication errors during transitions of care. One strategy that has emerged to improve data accuracy and close information gaps is the introduction of software applications and workflow models that allow patients to review, enter, and modify their own patient data (e.g. information about medications they are taking). Evaluating the quality and effectiveness of such patient-facing healthcare applications is critical, especially when this approach is applied to high-stakes clinical tasks such as medication reconciliation. In this paper we describe an approach that has been used to assess the usability of a patient-facing medication reconciliation and allergy review (MRAR) kiosk. The phases involved are described along with implications and challenges of carrying out this work.