Training tools using virtual reality (VR) are becoming more popular and cost-effective to develop and are increasingly adopted; yet there is no systematic means for evaluating their usability and pedagogical effectiveness. There are a wide range of training scenarios that can be scripted, from high level simulations of emergency response systems where participants using their avatars have to make complex decisions and communicate with each other, to low-level sensor-motor skills-based trainers where surgeons can practice suturing and cutting. We propose a classification framework for simulator-based training, associating each type of simulation with a specification of the types of skills it is designed to exercise and a corresponding evaluation plan. In this framework, objective measures involving task time and error rates can be formalized at the lower levels, and related subjective and objective measures can be identified at the top. Our framework is being implemented under the auspices of a recently funded New Media project in Canada (GRAND NCE) that spans two health training and simulation facilities (CSTAR and HSERC).