Suomen Akatemia  
Hakija / Yhteyshenkilö Zhao, Guoying
Organisaatio Oulun yliopisto
Tutkimusaihe Erottuvan modaliteetin oppiminen
Päätös 313600
Päätöspvm 10.10.2017
Rahoituskausi 01.01.2018 - 31.12.2019
Rahoitus (€) 306 950
Hankkeen julkinen kuvaus
Title: Separable Modality Learning Duration: 01.01.2018 to 31.12.2019 Site: University of Oulu. The research focuses on mimicking the human cognitive phenomenon of `multi-modal learning and missing modality in prediction' in designing computational models. In this project, we have initiated the new separable modality learning (SML) as a new sub-field of machine learning. Several publicly available multi-modal datasets are used for experimental evaluation and analysis. Three key problems are addressed: 1) How to effectively represent multi-modal data? 2) How to model the 'associate connection'? 3) How to make the output separable? Three main aspects are studied: 1) multi-modal fusion and cross-modal mapping; 2) building SML models for vector space with traditional learning; and 3) dealing with non-vector input with deep learning. The developed methods can provide general solutions and can be applied to any multi-modal application to boost both flexibility and accuracy.