Using Depth Image Based Rendering (DIBR) is the most popular method to generate virtual views for multi-view stereoscopic images or videos. However, when the depth values change rapidly, there are some hole occurrences in the region of the edges of the objects. This paper proposes a preprocessing method for the depth map by an edge-based asymmetric Gaussian blurring matrix before doing DIBR. The proposed depth preprocessing is based on edge direction of the edge of the objects; therefore, the proposed algorithm not only reduces the hole occurrences to enhance 3D virtual view quality but also preserves the shape of the objects in the foreground to keep the 3D view experience. The simulation results show that the boundaries of the objects are shape by using the proposed edge based asymmetric Gaussian blurring matrix than the other comparison methods. In addition, the proposed algorithm can keep the average depth value of the depth map to maintain more stereo vision for 3D view experience.