We propose a method for creating a semantic profile of user's interests emerging from pictures by application of neural networks trained for object recognition. We use BabelNet, an online encyclopaedic dictionary, to generalise object names into categories of interests. Our method is evaluated with ground-truth data based on social tagging mechanism. Experiments are conducted entirely on original data containing 60,000 images crawled from Flickr, evenly distributed among 300 users. Results show that object recognition methods combined with object category generalisation can be effectively used to predict user's interests. The accuracy of the presented method seems to change with the neural network used for object recognition (5 NN tested in total), therefore it has a strong potential for further development. ResNet-50 turned out to be the most accurate network in our experiment.
IOS Press, Inc.
6751 Tepper Drive
Clifton, VA 20124
Tel.: +1 703 830 6300
Fax: +1 703 830 2300 firstname.lastname@example.org
(Corporate matters and books only) IOS Press c/o Accucoms US, Inc.
For North America Sales and Customer Service
West Point Commons
Lansdale PA 19446
Tel.: +1 866 855 8967
Fax: +1 215 660 5042 email@example.com