Instruments and procedures continue to become more complex and challenging, but the display environments to which these technologies are connected have not kept pace. Display real‐estate (the size and resolution of the display), the configurability of the display, and the ability for display systems to incorporate, fuse and present diverse informational sources are limiting factors. The capabilities of display technologies are far exceeded by the procedures and instruments that rely on them.

In this paper we show how to break free from display constraints by moving forward with a hybrid, heterogeneous display framework that preserves key characteristics of current systems (low latency, specialized devices). We have engineered a hybrid display and are currently using it to build a surgical simulation and training environment within which we can evaluate both the technology and the performance of subjects using the technology.