This study proposes an improved optical flow algorithm based on semantic segmentation for real-time vehicle detection and tracking. The method combines the semantic segmentation capability of the SegmentAnything model with optical flow estimation technology. By generating precise vehicle region masks, it effectively narrows the search range for matching optical flow feature points, which not only improves the computation speed of optical flow but also enhances matching accuracy. Based on this, a real-time vehicle detection and tracking system is designed, including modules for multi-target detection, feature extraction, classification, and tracking. Experimental results show that this method outperforms existing methods in both detection accuracy and computational efficiency, making it suitable for real-time applications in complex traffic scenarios. It offers a new solution for intelligent transportation systems and autonomous driving technology.