Gesture recognition is a type of technology that can be used to control robots, manipulators, and other devices. Human-Computer Interaction (HCI) and research into creating simple-to-use interfaces by utilising natural communication have gotten a lot of press recently, as have human manipulation skills. When using gesture recognition, the user who is making the gesture is not necessary to wear any equipment or gadgets on their body. A camera linked to a device reads the body’s gestures and produces an output. The major purpose of this paper is to use MATLAB to control motor direction in proteus while also combining both software with VSPE (visual serial port emulator). Before immediately implementing this method on a robot, we can first simulate the outcomes to improve accuracy or correct models.