Deep learning is paid attention to by many researchers but it is not understandable because of its complex architecture and a black box of data processing. However, deep learning can construct appropriate features from raw data. On the other hand, statistical machine learning is understandable theoretically but needs features capturing characteristics of input data. If deep learning and statistical machine learning are combined, some efforts to construct machine learning decreases. In the paper a goal is to combine deep learning with statistical machine learning, support vector machines and to reduce manual setting efforts. To realize it a neural network constructs a kernel function and linear support vector machines constructs a discriminative hyperplane. In some classification tasks of UCI Machine Learning Repository the proposed method was evaluated. We confirmed the proposed method achieved the same performance as support vector machines without much adjustment.