Nowadays, the security of neural networks has attracted more and more attention. Adversarial examples are one of the problems that affect the security of neural networks. The gradient-based attack method is a typical attack method, and the Momentum Iterative Fast Gradient Sign Method (MI-FGSM) is a typical attack algorithm among the gradient-based attack algorithms. However, this method may suffer from the problems of excessive gradient growth and low efficiency. In this paper, we propose a gradient-based attack algorithm RMS-FGSM based on Root Mean Square Propagation (RMSProp). RMS-FGSM algorithm avoids excessive gradient growth by Exponential Weighted Moving Average method and adaptive learning rate when gradient updates. Experiments on MNIST and CIFAR-100 and several models show that the attack success rate of our approach is higher than the baseline methods. Above all, our generated adversarial examples have a smaller perturbation under the same attack success rate.