This large-scale study assesses the impact of human oversight on countering discrimination in AI-aided decision-making for sensitive tasks. It follows a mixed method approach, including a quantitative experiment with Human Resources (HR) and banking professionals in Italy and Germany (N=1411), and qualitative analyses through interviews and workshops with participants and fair AI experts. The results show that human overseers were equally likely to follow advice from a fair AI as from a generic, discriminatory AI. Human oversight does not prevent discrimination by the generic AI. Fair AI reduces gender bias but not nationality bias. Participants’ choices are neither more nor less responsive to their preferences when using an AI or when left on their own. Interviews and workshops with participants highlight individual, organizational and societal biases. In case of conflict, participants prioritize their company’s interests over their own view of fairness. Participants also ask for better guidance on when to override AI recommendations. Fair AI experts stress the need for a comprehensive approach when designing oversight systems. Both technological and social aspects should be taken into consideration to ensure fairness.