Deep neural networks have recently been used to address surface anomaly detection in industrial quality control and have achieved much success. However, addressing the data imbalance problem, especially the Easy/Hard Examples (EHE) imbalance problem, remain a challenging task in anomaly detection. To alleviate this problem, we propose a two-stage convolutional neural network with a novel loss function, i.e., concentrated loss function. Specifically, the concentrated loss function enables the model to pay more attention to hard examples and improve the quality of segmentation for imbalanced data. To verify the effect of our method, we implement our method on the surface anomaly detection dataset, i.e., the KolektorSDD2 dataset. The experimental results show the superiority of our method over the other state-of-the-art approaches.