The low resolution of infrared images makes it more difficult to detect objects, and the quality of detection results obtained by the CNN based object detection model are worse for few-shot problems. Two-stage Fine-tune Approach (TFA) is effective to improve the precision of detection for few-shot problems. Because of the category imbalance of training samples, TFA has the problem of misclassification. To solve this problem, TFA with similarity contrast (SC-TFA) is proposed. The VOVNetv2 is used as the backbone network to improve the detection accuracy. The similarity contrast detection head is added to the detection module to improving the classify performance. And both cosine similarity and Euclidean distance are used as the similarity measure in the contrast loss function. The effectiveness of the improved TFA for the few-shot problem is verified on the VOC dataset and the infrared ship dataset. The average precision of the novel categories (nAP) of SC-TFA on VOC dataset and the infrared ship dataset reaches 54.92% and 41.1% respectively, which is 4.7% and 3.4% higher than TFA.