Conference Proceeding Article
Online reviews are prevalent. When recounting their experience with a product, service, or venue, in addition to textual narration, a reviewer frequently includes images as photographic record. While textual sentiment analysis has been widely studied, in this paper we are interested in visual sentiment analysis to infer whether a given image included as part of a review expresses the overall positive or negative sentiment of that review. Visual sentiment analysis can be formulated as image classification using deep learning methods such as Convolutional Neural Networks or CNN. However, we observe that the sentiment captured within an image may be affected by three factors: image factor, user factor, and item factor. Essentially, only the first factor had been taken into account by previous works on visual sentiment analysis. We develop item-oriented and user-oriented CNN that we hypothesize would better capture the interaction of image features with specific expressions of users or items. Experiments on images from restaurant reviews show these to be more effective at classifying the sentiments of review images.
review images, convolutional neural networks, visual sentiment analysis
Databases and Information Systems | Numerical Analysis and Scientific Computing
Data Management and Analytics
MM '17: Proceedings of the ACM Multimedia Conference, Mountain View, CA, October 23-27
City or Country
TRUONG, Quoc Tuan and LAUW, Hady Wirawan.
Visual sentiment analysis for review images with item-oriented and user-oriented CNN. (2017). MM '17: Proceedings of the ACM Multimedia Conference, Mountain View, CA, October 23-27. 1274-1282. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/3885
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.