As the increase of digital image resources, image retrieval has been received widespread research interest. A popular approach for realizing the retrieval of relevant images from an image database is to match the vision features like histogram, color layout, textures and shapes automatically derived from images. However, the visual similarity does not always match to the human required retrieval results. This problem is known as the gap between visual similarity and human semantic. In this paper, we represent a method to bridge the gap. In our method, first, image's edges and their relative position information are derived. After that, independent factors hidden in the derived edge and position information are extracted by using a mathematic method referred to as the Singular Value Decomposition (SVD). We present our analysis on the relationship between the extracted independent factors and the human semantic. The most important contribution of this paper is that most extracted independent factors based on our method are demonstrated to be related to human semantic according to our experiments which are performed on 7,000 images.