As one of the most commonly performed neurosurgical procedures, ventriculostomy training simulators are becoming increasingly familiar features in research institutes and teaching facilities. Despite their widespread implementations and adoption, simulators to date have not fully explored the landscape of performance metrics that reflect surgical proficiency. They opt instead for measures that are qualitative or simple to compute and conceptualize. In this paper, we examine and compare the use of various metrics to characterize the performance of users on simulated part-task ventriculostomy scenarios derived from patient data. As an initial study, we examine how our metrics relate to expert classification of scenario difficulty as well as measures of anatomical variation.