Chart checking is a time intensive process with high cognitive workload for physicists. Previous studies have partially automated and standardized chart checking, but limited studies implement data-driven approaches to reduce cognitive workload for quality assurance processes. This study aims to evaluate feature selection methods to improve the interpretability and transparency of machine learning models in predicting the degree of difficulty for a pretreatment physics chart check. We compare chi-square, mutual information, feature importance thresholding, and greedy feature selection for four different classifiers. Random forest has the highest performance with SMOTE oversampling using mutual information for feature selection (accuracy 84.0%, AUC 87.0%, precision 80.0%, recall 80.0%). This study demonstrates that feature selection methods can improve model interpretability and transparency.