Q-Learning is one of the well-known model-free RL methods. When it discovers a good reward, it needs much iteration to diffuse the information. In this paper, we used an adaptive model learning method based on tree structure to imitate experience. Predictions let agent have extra experience to learn policy indirectly. Agent can use the tree-model to imitate the transition between two states. However, when model is not accurate, it may get through absorbing states. Getting through absorbing states influence the learning efficiency. In order to solve this problem, we propose two methods to enhance the tree-model. For demonstrating the proposed method, we introduce two simulations to verify the proposed methods. The simulation results demonstrate that the training rate of our method can improve obviously.