Based on the generative adversarial network, this paper proposes a method of colorizing gray images without supervision and applies it to the field of the automatic coloring of gray comic sketches to solve the problem of the high production cost of color comics and the large consumption of human resources. The method proposed in this paper mainly improves the classical generative adversarial network model in the following aspects: first, the idea of the residual network is added to the builder model and the discriminator model to ensure that the training of the model can develop in the right direction; secondly, add a gradient penalty term to the loss function of the discriminator to accelerate the convergence of the model and speed up the iteration; at the same time, the activation function in the original model is changed to the Mish activation function, so that the information flowing through the network model has higher accuracy and better generalization. This paper trained the model of the Anime Sketch Colorization Pair dataset [1] from Kaggle, and the final experimental results show that the method is practical and feasible.