Regression trackers have been shown to perform superiorly in visual tracking. However, existing researches in regression trackers mainly explore deep models for feature extraction, and then use sophisticated architectures for online detection. Such systems should optimize a massive number of trainable parameters. In this paper, we present a simple yet effective visual tracking system, called LiteCNT. Our algorithm only consists of three convolutional layers for the whole tracking process. In addition, a multi-region convolutional operator is introduced for regression output. This idea is simple but powerful as it enables our tracker to capture more details of target object. We further derive an efficient and effective operator to approximate multi-region aggregation.