As one interpretable technique, fuzzy neural network (FNN) can be equipped into any deep models, but it faces the problem of high dimension when FNN is used for designing deep networks. To break the curse of dimensionality, in this paper, we use regularization method to reduce the influence of dimensionality on FNN, and based on L_2,1 norm (Group Lasso) two different regularizer terms are designed. Using the gradient method, FNN can learn to evaluate the importance of features and rules, respectively, and then realizes feature selection (FS) and rule generation (RG). By dint of the simulation results on the benchmark classification problems Iris and Sonar, the validity of our proposed fuzzy classifier is verified, i.e., without decreasing the classification performance, the structure of the fuzzy model can be simplified. The regularized FNN can be easily used for interpretable deep model design.