Image recognition using deep learning, especially deep convolution neural network (DNN), of great success to human due to increasing use of computer vision in our daily life. In this paper, we introduce a novel framework of image recognition system based on quaternion fractional-order radial orthogonal moments and deep learning. The proposed image recognition system is derived by combining quaternion fractional-order polar harmonic-Fourier moments (QFr-PHFMs) and Micro-Convolution Neural Network (Micro-CNN). The proposed methodology can use a small number of network layers and achieve high-quality recognition accuracy, especially in the case of image processing with high noise and smooth filtering conditions. Theoretical analyses and experimental results showed that the proposed methodology offer enhanced image recognition compared with the different moment-based feature-extraction algorithms and the existing CNN methods.