With the rapid development of artificial intelligence technology, research on autonomous driving technology is becoming increasingly popular. In the autonomous driving panoramic imaging system, it is necessary to use image stitching technology to perform real-time stitching of image information collected by multiple cameras around the vehicle body. Therefore, requirements for image data processing include large data volume, high speed, and low power consumption. The parallel computing characteristics of Field Programmable Gate Array (FPGA) can accelerate key image stitching algorithms. This article focuses on hardware acceleration of the RANSAC (Random Sample Consensus) algorithm in image stitching. The algorithm is designed in parallel using high-level synthesis (HLS) technology and optimized accordingly to find a suitable method for algorithm optimization. Comparing the FPGA algorithm circuit with the Open CV algorithm program, the hardware running time of the algorithm is only 2.816ms, and the processing speed is 38 times faster than the latter. The RANSAC algorithm implemented by FPGA meets the requirements of real-time image processing and can be applied in real-time image stitching systems.