In human smart nutrition systems, environment based food classification has become popular to help analyzing the food intake based on the nutrition related activity. In this paper, we address the problem of food related environments, which refer to different eating places such as, bars, restaurants, coffee shops, etc. using state-of-the-art convolutional neural networks (CNNs). We collected a new dataset on different food related environments by integrating three publicly available datasets: Places365, ImageNet and SUN397. We have named it “FoodPlaces” and it contains 35 different types of classes. In order to achieve satisfactory results on the food related environment recognition, we fine-tuned several state-of-the-art CNNs, such as VGG16, RsNet50 and InceptionV3 using different transfer learning approaches. The results show that the fully fine-tunned InceptionV3 yields 75.22% classification accuracy among the discussed state-of-the-art CNNs.