Deep learning combined with autonomous drones is increasingly seen as an enabler of automated aircraft inspection which can support engineers detect and classify a wide range of defects. This can help increase the accuracy of damage detection, reduce aircraft downtime, and help prevent inspection accidents. However, a key challenge in neural networks is that their stability is not yet well understood mainly due to the large number of dimensions and the complexity of their shapes. This paper illustrates this challenge through a use case that applies MASK R-CNN to detect aircraft dents. The results show that environmental factors such as raindrops can lead to false positives. The paper also proposes various test scenarios that need to be considered by the developers of the drone-based inspection concept to increase its reliability.