Abstract: Caricature face recognition is an interesting but also difficult task due to the huge exaggeration between two different face modalities, photos, and caricatures. Therefore, we propose a new representation for recognition that is fused by the representation learned from photos, caricatures, and generated faces. Each generated face contains four main facial components. Photos, caricatures, and generated faces are sent to Photo-ResNet, Caricature-ResNet, and Generated-ResNet to learn specific representations. Then, the learned three representations are sent to a fully connected layer. We adopt Softmax loss and Center Loss for training, which can reduce the distance of intra-class. To test the performance of our proposed representation, we build a new dataset for caricature face recognition, which consists of 259 subjects, with 6490 caricatures and 8143 photos. The dataset we build is the biggest available caricature dataset. Several basic methods are used for caricature face recognition. To test the discrimination of our proposed representation, two more experiments are fulfilled, including searching photos according to the selected caricature (CTP) and searching caricatures according to the selected photo (PTC), and our proposed method performs better than other convolutional neural network (CNN)-based representations.

Key words: caricature face recognition, generated face, facial components, Center Loss, caricature dataset