The aim of transfer learning is to reuse learnt knowledge across different contexts. In the particular case of cross-domain transfer (also known as domain adaptation), reuse happens across different but related knowledge domains. While there have been promising first results in combining learning with symbolic knowledge to improve cross-domain transfer results, the singular ability of ontologies for providing classificatory knowledge has not been fully exploited so far by the machine learning community. We show that ontologies, if properly designed, are able to support transfer learning by improving generalization and discrimination across classes. We propose an architecture based on direct attribute prediction for combining ontologies with a transfer learning framework, as well as an ontology-based solution for cross-domain generalization based on the integration of top-level and domain ontologies. We validate the solution on an experiment over an image classification task, demonstrating the system’s improved classification performance.