In the aircraft industry, where high precision geometric control is vital, unexpected component deformation, due to the release of internal residual stress, can limit geometric accuracy and presents process control challenges. Prediction of component deformation is necessary so that corrective control strategy can be defined. However, existing prediction methods, that are mainly based on the prediction or measurement of residual stress, are limited and accurate deformation prediction is still a research challenge. To address this issue, this paper presents a data-driven method for deformation prediction based on the use of in-process monitored deformation data. Deformation, which is caused by an unbalanced internal residual stress field, can be accurately monitored during the machining process via an instrumented fixture device. The state of the internal stress field within the part is first estimated by the using the part deformation data collected during machining process, and then, the deformation caused by a subsequent machining process is predicted. Deep learning is used to establish the estimating module and predicting module. The estimating module is used to infer the unobservable residual stress field as vectors by using sparse deformation data. The inferred vector is then used to predict the deformation in the predicting module. The proposed method provides an effective way to predict deformation during the machining of monolithic components, which is demonstrated experimentally.