Real-time tracking of non-rigid objects for use in interfaces of VR-simulators is presented. Markers are attached to the objects and observed by several cameras with integrated image-processing hardware which extracts relevant marker data (centroid, area & color) in real-time. Data from the different cameras is then matched in the host PC to reconstruct the 3D positions. We present two approaches to this special matching problem because standard image feature based algorithms are not feasible for marker-based tracking. A model of the deformation is extracted from the reconstructed 3D point cloud and the simulation model is updated accordingly. Experiments with a prototype of a deformable eye interface for the ophthal-mosurgical simulator EYESI show that latency, robustness and accuracy of the deformation tracking are adequate for application in VR simulators. The approach is extensible to other types of simulators where deformable tissue has to be tracked.