External validity is the extent to which the findings of an experimental study are applicable or can be generalized to other people and contexts. External validity includes both population validity (i.e., the representativeness of the sample) and ecological validity (i.e., the representativeness of other contextual variables such as the stimuli, workflow, and environment). There is no consensus on the number and names of the ecological validity dimensions researchers should consider when designing or evaluating usability evaluations in health care. Therefore, in this paper, we integrated concepts from 3 ecological validity frameworks into a unified external validity. We used this new framework to describe the dimensions of external validity of a previous usability study. This framework can inform the design and description of usability evaluations to enhance reproducibility and comparison of findings and ultimately better understand the impact of different dimensions and external validity holistically.