Breast cancer is one of the most dangerous diseases that threaten women. Computer-aided diagnosis systems can be used to analyse breast images in order to detect breast tumours early. This paper proposes a new method for feature learning from breast cancer images. The learned features are used to discriminate between malignant and benign tumours. We marry the traditional feature extraction methods with deep learning approaches, in the sense that we automatically learn local descriptors from the input images without hand-crafting the feature extraction methods. The proposed technique uses a bio-inspired optimization method, called whale optimization algorithm, to learn local descriptors from the input images. The learned features are tuned to the input images and can describe the local/global arrangement of pixels in them. Unlike the usual deep learning approaches, small datasets can be used to train the proposed method. To validate the proposed feature learning method, we used two breast cancer datasets: mammographic and ultrasound images for benign and malignant cases. The experimental results demonstrate that the proposed method gives good classification results compared with the state of the art ones.