This paper describes a robot system which can follow a specific person while avoiding obstacles and other people walking around. This system consists of various functional modules, which are a stereo-based robust person detection and tracking, a laser range finder-based map generation, and an on-line randomized motion planning, implemented on a self-contained wheeled mobile robot. All these modules are implemented using RT-middleware, which enhances software development, maintenance, and reusability. The implemented system is tested in a reasonably complex environment with several walking people at a time. The experimental results show the feasibility of the system.