Portrait matting refers to separating the portrait part from the background in an image. The difficulty of the problem lies in accurately identifying the pixels of the person and also maintaining the contour details. In this paper, we propose a fully automatic deep learning approach to achieve portrait matting. Firstly, semantic segmentation is used to predict the probabilities of pixels belonging to portrait, background, and unknown region, then a trimap is obtained. In order to remove the misclassification of pixels, we refine the portion of head contour for the trimap. The method used is to introduce the result of facial landmark detection, and erosion operation is performed on the head region while maintaining the integrity of the facial contour of the portrait. After that, we use deep matting method to predict the alpha value in the image to get the matting results at the details. We then propose a novel framework that integrates the optimised trimap, the deep matting result, and the original image to obtain the final matting result. Both qualitative and quantitative experiments verify the effectiveness of the proposed method.