Academy of Finland
Funding decision
Name Zhao, Guoying
Organisation OY
Project title
Separable Modality Learning
Decision No. 313600
Decision date 10.10.2017
Funding period 01.01.2018 - 31.12.2019
Funding 306 950
WebFOCUS Report
Project description
Title: Separable Modality Learning Duration: 01.01.2018 to 31.12.2019 Site: University of Oulu. The research focuses on mimicking the human cognitive phenomenon of `multi-modal learning and missing modality in prediction' in designing computational models. In this project, we have initiated the new separable modality learning (SML) as a new sub-field of machine learning. Several publicly available multi-modal datasets are used for experimental evaluation and analysis. Three key problems are addressed: 1) How to effectively represent multi-modal data? 2) How to model the 'associate connection'? 3) How to make the output separable? Three main aspects are studied: 1) multi-modal fusion and cross-modal mapping; 2) building SML models for vector space with traditional learning; and 3) dealing with non-vector input with deep learning. The developed methods can provide general solutions and can be applied to any multi-modal application to boost both flexibility and accuracy.