We develop a hybrid machine learning architecture, the Influence Relevance Voter (IRV), where an initial geometry- or kernel- based step is followed by a feature-based step to derive the final prediction. While other implementations of the general idea are possible, we use a k-Nearest-Neighbor approach to implement the first step, and a Neural Network approach to implement the second step for a classification problem. In this version of the IRV, the rank and similarities of the k nearest neighbors of an input are used to compute their individual relevances. Relevances are combined multiplicatively with the class membership values to produce influences. Finally the influences of all the neighbors are aggregated to produce the final probabilistic prediction. IRVs have several advantages: they can be trained fast, they are easily interpretable and modifiable, and they are not prone to overfitting since they rely on extensive weight sharing across neighbors. The IRV approach is applied to the problem of predicting whether a given compound is active or not with respect to a particular biochemical assay in drug discovery and shown to perform well in comparison to other predictors. In addition, we also introduce and demonstrate a new approach, the Concentrated ROC (CROC), for assessing prediction performance in situations ranging from drug discovery to information retrieval, where ROC curves are not adequate, because only a very small subset of the top ranked positives is practically useful. The CROC approach uses a change of coordinates to smoothly magnify the relevant portion of the ROC curve.