Automatically identifying characters in movies has attracted researchers' interest and led to several significant and interesting applications. However, due to the vast variation in character appearance as well as the weakness and ambiguity of available annotation, it is still a challenging problem. In this paper, we investigate this problem with the supervision of actor-character name correspondence provided by the movie cast. Our proposed framework, namely, Cast2Face, is featured by: 1) we restrict the assigned names within the set of character names in the cast; 2) for each character, by using the corresponding actor and movie name as keywords, we retrieve from the Google image search and get a group of face images to form the gallery set; 3) the probe face tracks in the movie are then identified as one of the actors by a robust kernel multitask joint sparse representation and classification method; and 4) the conditional random field model with consideration of the constraints between face tracks is introduced to enhance the final labeling. Finally, the assigned actor name of a face track is then mapped to the character name based on the cast again. Besides face naming, we further apply the proposed method to spotlight the summarization of a particular actor in his/her movies. We conduct extensive experiments and empirical evaluations on several feature-length movies to demonstrate the satisfying performance of our method.
Cast analysis, character identification, conditional random field (CRF), face recognition, multitask learning
Computer Sciences | Databases and Information Systems | Graphics and Human Computer Interfaces
Data Management and Analytics
IEEE Transactions on Circuits and Systems for Video Technology
Institute of Electrical and Electronics Engineers (IEEE)
GAO, Guangyu; XU, Mengdi; SHEN, Jialie; MA, Huangdong; and YAN, Shuicheng.
Cast2Face: Assigning character names onto faces in movie with actor-character correspondence. (2016). IEEE Transactions on Circuits and Systems for Video Technology. 26, (12), 2299-2312. Research Collection School Of Information Systems.
Available at: http://ink.library.smu.edu.sg/sis_research/3535
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.