Bayesian models are a useful tool to propagate the rational implications of human beliefs expressed as probabilities. They can yield surprising, counterintuitive and, if based on valid models, useful results. However, human users can be reluctant to accept their results if they are unable to find explanations providing clear reasons for how and why they were arrived at, which existing explanation methods struggle with. This is particularly important in the legal domain where explanatory justifications are as important as the result and where the use of Bayesian models is controversial. This paper presents a novel approach to explain how the outcome of a query of Bayesian network was arrived at. In the process, it augments the recently developed support graph methodology and shows how support graphs can be integrated with qualitative probabilistic reasoning approaches. The usefulness of the approach is illustrated by means of a small case study, demonstrating how a seemingly counterintuitive Bayesian query result can be explained with qualitative arguments.