We describe two recent research projects: the Virtual Hand Laboratory, and Remote Manipulation in Endoscopic Surgery. The Virtual Hand Laboratory (VHL) is a prototype experimental tool for investigating human visuomotor coordination for object manipulation in augmented and virtual environments. The Remote Manipulation in Endoscopic Surgery (RMES) project examined surgeon's use of viewing and manipulating technologies in laparoscopic surgery, both in clinical and experimental settings. Current research brings together these two parallel research projects (VHL and RMES), for applications in planning and real-time execution of surgical procedures as well as for surgical training. We outline our research directions and detail current activities on superposition of display space on the workspace for the surgeon’s hands.