The nearest neighbour (NN) classifier is often known as a ‘lazy’ approach but it is still widely used particularly in the systems that require pattern matching. Many algorithms have been developed based on NN in an attempt to improve classification accuracy and to reduce the time taken, especially in large data sets. This paper proposes a new classification technique based on k-Nearest Neighbour (k-NN), called k-Nearest & Farthest Neighbours (k-NFN). Farthest neighbours are used to identify classes that an unseen record may not belong to and are considered with the nearest neighbours in the classification decision. Two neighbour voting systems are also proposed to further improve k-NN and k-NFN accuracy. The first uses a ranking system and the second uses a spectrum to consider how near or far a neighbour actually is. The accuracy of our three proposed k-NFN techniques and k-NN are compared using the standard ten cross fold validation experiments on a number of real data sets, evidencing the superiority of our proposed techniques in terms of accuracy.