There are well-known issues in eliciting probabilities, utilities, and criteria weights in real-life decision analysis. In this paper, we examine automatic multi-criteria weight-generating algorithms which are seen as one remedy to some of the elicitation issues. The results show that the newer Sum Rank approaches perform better in terms of both performance and robustness than older (classical) methods, also when compared to the new and promising geometric class of methods. Additionally, as expected the cardinal surrogate models perform better than their ordinal counterparts (with one exception) due to their ability to take more information into account. Unexpectedly, though, the well-established linear programming model’s performance is worse in this respect than previously thought, despite a promising mapping between linear optimisation and surrogate weight generation which is explored in the paper.