In the last years, several methodologies for ontology engineering have been proposed. Most of these methodologies guide the engineer from a first paper draft to an implemented – mostly description logics-based – ontology. A quality assessment of how accurately the resulting ontology fits the initial conceptualization and intended application has not been proposed so far. In this paper, we investigate the role of semantic similarity as a quality indicator. Based on similarity rankings, our approach allows for a qualitative estimation whether the domain experts' initial conceptualization is reflected by the developed ontology and whether it fits the users' application area. Our approach does not propose yet another ontology engineering methodology but can be integrated into existing ones. A plug-in to the Protégé ontology editor implementing our approach is introduced and applied to a scenario from hydrology. The benefits and restrictions of similarity as a quality indicator are pointed out.