Prolonged use of a keyboard or mouse can lead to continuous bending and twisting of the wrist, which can lead to muscle fatigue and pain. Finger exercises can reduce the hand muscle fatigue effectively. However, the repetitive exercises will bring boring and tedious experience. In this paper, we developed a hand rehabilitation exercise system based on gesture recognition technology and gamification design, which provides a more enjoyable, personalized and effective rehabilitation experience. The hand fatigue relief system was designed with the fusion of game elements and gesture interaction, which can enhance users’ engagement and their rehabilitation motivation.