Minimum spanning tree (MST) has been devised for non-local cost aggregation to solve the stereo matching problem. However, the cost aggregation is employed directly from leaf toward root node, then in an inverse pass without considering any decision rules. And a small amount of noise is also existed in stereo image pairs. Both of the limitations often lead to failure in achieving more competitive results. This paper presents a novel stereo matching algorithm using forward-backward diffusion and pruning-based cost aggregation. In “forward-backward” process, the raw image pairs are smoothened on a horizontal tree structure as well as retaining image edges sharp. During cost aggregation, the MST where a complete graph involves the whole image pixels is cut off self-adaptively when the depth edge information is referred to. Each node in this tree receives supports from all other nodes which belong to similar depth regions. Meanwhile, an enhanced edge similarity function between two nearest neighboring nodes is formulated to deal with the small-weight-accumulation problem in textureless regions. Consequently, the cost volume can be well aggregated. The proposed method is demonstrated on Middlebury v.2 & v.3 datasets and can obtain good performance in disparity accuracy compared with other five MST based stereo matching methods.