Semantic Image-Based Profiling of Users' Interests with Neural Networks

Szymon Wieczorek , Dominik Filipiak , Agata Filipowska


We propose a method for creating a semantic profile of user's interests emerging from pictures by application of neural networks trained for object recognition. We use BabelNet, an online encyclopaedic dictionary, to generalise object names into categories of interests. Our method is evaluated with ground-truth data based on social tagging mechanism. Experiments are conducted entirely on original data containing 60,000 images crawled from Flickr, evenly distributed among 300 users. Results show that object recognition methods combined with object category generalisation can be effectively used to predict user's interests. The accuracy of the presented method seems to change with the neural network used for object recognition (5 NN tested in total), therefore it has a strong potential for further development. ResNet-50 turned out to be the most accurate network in our experiment.
Author Szymon Wieczorek
Szymon Wieczorek,,
, Dominik Filipiak (UEP)
Dominik Filipiak,,
- Poznań University of Economics and Business
, Agata Filipowska (WIiGE / KIE)
Agata Filipowska,,
- Department of Information Systems
Publication size in sheets0.55
Book Demidova Elena, Zaveri J. Amrapali, Simperl Elena (eds.): Emerging Topics in Semantic Technologies : ISWC 2018 Satellite Events, Studies on the Semantic Web, vol. 36, 2018, IOS Press, ISBN 978-1-61499-893-8, [978-1-61499-894-5], 254 p.
Keywords in Englishonline social networks user characteristics activity recognition and understanding semantic tagging deep learning
Languageen angielski
Score (nominal)140
Score sourceconferenceList
ScoreMinisterial score = 140.0, 22-04-2020, ChapterFromConference
Citation count*3 (2020-09-25)
Share Share

Get link to the record

* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or Perish system.
Are you sure?