With the rapid growth of image and video data as well as the fast spread of user-generated content in social media and cloud services, it has become increasingly difficult for users to have efficient access and effective management of their digital content. In this paper we present a novel integrated open source multimedia content management and access framework, called VisualLabel, that enables smart photo services based on automated visual content analysis, annotation, search and retrieval using state of the art analysis back ends for services such as Facebook and Flickr. This paper includes detailed descriptions of the high-level architecture used in the VisualLabel framework and proof-of-concept implementations of a front-end service, along with three analysis back ends and a web client, all of which demonstrate the basic functionality provided by the framework.