A memory-friendly multi-modal emotion analysis for smart toy
Author(s)
Poon, Geoffrey
Pang, Raymond Wai Man
Author(s)
Li, K.-M.
Date Issued
2017
Publisher
IEEE
Related Publication(s)
Proceedings of 2017 IEEE International Symposium on Multimedia (ISM 2017)
Start page
432
End page
437
Abstract
The recent advance in deep learning technologies has provided many opportunities to the industries in developing smarter products, such as smart toys, smart cars and smart homes. Unfortunately, a common practical issue to these deep learning methods is the high requirements in computing power even for the prediction part. As the deep learning models' complexities increase, the memory requirements for these models usually exceed the limit of many low end computing devices, such as mobile IoT devices. It is even worse when a multimodel approach is employed. In this paper, we will demonstrate with the development of multimodal emotion analysis on a smart toy, where the user's speech, facial expression and action are used for the understanding of user's emotion. By trimming down DNN complexity or replaced by other learning approaches, we are able to sequeeze four classifiers in 800MB of memory. Finally, results of these methods are ensembled with a fusion approach using a fully connected neural network to obtain a more accuracy and stable result. Our multimodal approach achieved an improvement of about 20% when comparing any unimodal emotion analysis.
SFU Affiliated Publication
Yes
Availability at SFU Library
No database links found.