Standard linear models are very easily readable, but have limited model flexibility. Advanced neural network models and kernel-based learning techniques are less straightforward to interpret but can capture more complex multivariate non-linear relations. Whereas more flexible models may be appealing because of the higher learning capacity, it is more challenging to control the generalization capacity and avoid overfitting using, e.g., Bayesian inference or model complexity criteria. In financial practice, it is important to consider the prediction capacity combined with the element of model risk, inherent in so-called black box models. Model combinations using linear and kernel models and rule extraction techniques are used to combine kernel models with higher performance and limit model risk. The approach is illustrated with practical case studies.