In the field of trajectory prediction, when new scene data is continuously introduced, the system may forget the historical trajectory information that has been learned before, leading to the phenomenon of decreasing the prediction accuracy of previous scenes, which is called catastrophic forgetting. The Multi-Memo Net framework proposed in this paper is an instance-based approach that stores typical trajectories through a memory bank thereby establishing a direct link between the current and historical scenes. During training, self-encoders and decoders are used to efficiently memorize the feature expressions of past trajectories and future intentions; during prediction, the controller finds similar scenes that have occurred in history in the memory bank based on the past trajectories and contextual information for trajectory prediction, and ultimately obtains a multimodal trajectory prediction through K-means clustering, and while predicting, the model can keep updating the content of the memory bank to learn, thus iteratively optimizing the model prediction performance. Thus, the model prediction performance is iteratively optimized. Compared with baseline models such as Kalman filter and Memory Augmented Networks for Multiple Trajectory Prediction (MANTRA) model, the method in this paper has significant improvement in Average Displacement Error (ADE) and Final Displacement Error (FDE) metrics.