Stroke patients with hemiplegia are often accompanied by balance disability, which has a serious impact on their life, work and rehabilitation treatment. In order to estimate their recovery and formulate a rehabilitation plan, the Berg Balance Scale (BBS) is usually used to evaluate their balance disability. At present, BBS is often evaluated manually. But the human resources are tight in the hospital, and it is difficult for patients to participate regularly. In order to reduce the burden of both doctors and patients, this paper uses artificial intelligence algorithm to automatically score the test process of Berg Balance Scale. Firstly, the method uses HRNet (High-Resolution net) algorithm to obtain the 2D human bone information of patients in the video sequence. Then, MHFormer (Multi-Hypothesis transformer) network was used to predict the 3D bone information of patients. Finally, based on the features of human joint points, human motion recognition and quantitative scoring were carried out. The experimental results showed that the same effect could be achieved by using intelligent algorithm to score Berg scale.