This page summarizes all the activities we have developed in the context of CAPES STIC-AmSud project (2024-2025).

Multimodal/Multiview Learning Applied to Real Problems (MMLARP)

Abstract

We perceive the world around us in a multimodal way. Consequently, multimodal systems have received much attention in Artificial Intelligence (AI). The challenges in such an approach go far beyond the simple fusion of modalities. In practice, it is relevant to consider: a) the modalities to be used, b) the representation of each modality, c) the way modalities interact, and d) how learning accounts for different modalities (individually or simultaneously). This proposal aims to enhance our understanding of the strategies above and develop novel multimodal and multiview predictive models, investigating alternatives for representing and combining different modalities in Pattern Recognition and Machine Learning. The project specifically focuses on researching and developing solutions using multimodal prediction models, emphasizing the architecture of these models, new strategies for generating representations for each modal (views), and new strategies for the selection/fusion of views and modalities considering possible intermodal relationships. Additionally, the project comprises the investigation of strategies to improve the training process of multiple modalities. Each stage of the working plan has the potential to result in publications in qualified international scientific journals and conferences. The project's social contribution relates to human resource formation and the potential for generating technological innovations that can positively impact society in terms of emotion analysis to help in the diagnosis of depression, medical image analysis, document image retrieval in digital libraries, and car safety (driver assistance systems). Ultimately, this proposal consolidates scientific exchange and collaboration of research teams from France, Brazil, and Chile, started in SticAmsud 2014.

Institutions

  • Université de Rouen (UR, Rouen, France)
  • Pontificia Universidade Católica do Paraná (PUCPR, Curitiba, Brazil)
  • Universidade Federal do Paraná (UFPR, Curitiba, Brazil)
  • Universidade Federal de Pernambuco (UFPE, Recife, Brazil)
  • Universidad de los Andes (UA, Santiago, Chile)

Team

  • Prof. Alceu de Souza Britto Jr (Coordinator, PUCPR/Brazil)
  • Prof. Jean Paul Barddal (PUCPR/Brazil)
  • Prof. André Gustavo Hochuli (PUCPR/Brazil)
  • Carlos Magno (PhD student, PUCPR/Brazil)
  • Caio S. Dias (PhD student, PUCPR/Brazil)
  • Luciana Menon (PhD student, PUCPR/Brazil)
  • Daniel Vriessman (PhD student, PUCPR/Brazil)
  • Cristiano Garcia (PhD student, PUCPR/Brazil)
  • Prof. Luiz Eduardo S. Oliveira (Associate Coordinador, UFPR/Brazil)
  • Everton Baro (PhD student, UFPR/Brazil)
  • Leandro Ensina (PhD student, UFPR/Brazil)
  • Prof. George D. C. Cavalcanti (Associate Coordinator, UFPE/Brazil)
  • Francimaria R. S. Nascimento (PhD student, UFPE/Brazil)
  • Juscimara G. Avelino (PhD student, UFPE/Brazil)
  • Keila Santos (PhD student, UFPE/Brazil)
  • Livia Alexandre (PhD student, UFPE/Brazil)
  • Lucas Amorim (PhD student, UFPE/Brazil)
  • Cynthia Moreira Maia (PhD student, UFPE/Brazil)
  • Prof. Laurent Heutte (French Coordinator, Université de Rouen/France)
  • Prof. Simon Bernand (Université de Rouen/France)
  • Profa. Stéphane Nicolas (Université de Rouen/France)
  • Joseph Assaker (PhD student, Université de Rouen/France)
  • Prof. Jose M. Saavedra (Chilean Coordinator, Universidad de los Andes/Chile)

Links to the Main Events