We consider the classes of Markov chains, Markov decision processes, and stochastic games definable by pushdown automata with probabilistic and non-deterministic choice. We give an overview of the existing results and present selected concepts and techniques that have been used to establish these results.