Publication Type

Journal Article

Version

acceptedVersion

Publication Date

4-2019

Abstract

To simulate the concept acquisition and binding of different senses in the brain, a biologically inspired neural network model named perception coordination network (PCN) is proposed. It is a hierarchical structure, which is functionally divided into the primary sensory area (PSA), the primary sensory association area (SAA), and the higher order association area (HAA). The PSA contains feature neurons which respond to many elementary features, e.g., colors, shapes, syllables, and basic flavors. The SAA contains primary concept neurons which combine the elementary features in the PSA to represent unimodal concept of objects, e.g., the image of an apple, the Chinese word “[píng guǒ]” which names the apple, and the taste of the apple. The HAA contains associated neurons which connect the primary concept neurons of several PSA, e.g., connects the image, the taste, and the name of an apple. It means that the associated neurons have a multimodal response mode. Therefore, this area executes multisensory integration. PCN is an online incremental learning system, it is able to continuously acquire and bind multimodality concepts in an online way. The experimental results suggest that PCN is able to handle the multimodal concept acquisition and binding effectively.

Keywords

Concept acquisition and binding, multimodal learning, online incremental learning, perception coordination network (PCN), unsupervised learning

Discipline

OS and Networks | Software Engineering

Research Areas

Data Science and Engineering

Publication

IEEE Transactions on Neural Networks and Learning Systems

Volume

30

Issue

4

First Page

1104

Last Page

1118

ISSN

2162-237X

Identifier

10.1109/TNNLS.2018.2861680

Publisher

IEEE

Copyright Owner and License

Authors

Additional URL

https://doi.org/10.1109/TNNLS.2018.2861680

Download

Share

COinS