Inspection of multi-layered riveted structures of aircraft and detection of subsurface cracks under fastener head is a major challenge in aviation industry. Magnetic field image (MFI) obtained with high sensitivity sensor array is widely used for inspection of this kind of defects. The magnetic field image contains defect information, from which the defect can be identified. Furthermore, in order to quantitatively analyze the degree of hazard of the defect to the aircraft, it is important to know the actual dimensions of the defects. In this paper, a method based on error back propagation (BP) training artificial neural networks (ANN) is proposed to estimate the defect dimensions from MFI. The output error of the network is decreased along the gradient direction by intelligently adjusting the connection strength and the threshold of the ANN using gradient descent method. It is demonstrated that the BP neural networks have a high accuracy for quantifying defects in multilayer rivet structure.