This paper presents a new paradigm for imagery analysis where imagery is annotated using terms defined in ontologies, enabling more powerful querying and exploitation of the analysis results. The ontology terms represent the concepts and relationships necessary to effectively describe the objects and activities within a domain of interest. A platform for viewing and editing imagery annotations is described along with a specialized semantic knowledge base capable of efficiently querying the information using semantic, spatial, and temporal qualifiers. The ontologies used for representing the annotations and domain of interest are also described.