We propose a modified version of the U-Net architecture for segmenting and classifying brain tumors, introducing another output between down- and up-sampling. Our proposed architecture utilizes two outputs, adding a classification output beside the segmentation output. The central idea is to use fully connected layers to classify each image before applying U-Net’s up-sampling operations. This is achieved by utilizing the features extracted during the down-sampling procedure and combining them with fully connected layers for classification. Afterward, the segmented image is generated by U-Net’s up-sampling process. Initial tests show competitive results against comparable models with 80.83%, 99.34%, and 77.39% for the dice coefficient, accuracy, and sensitivity, respectively. The tests were conducted on the well-established dataset from Nanfang Hospital, Guangzhou, China, and General Hospital, Tianjin Medical University, China, from 2005 to 2010 containing MRI images of 3064 brain tumors.