MMLARP MMLARP
Active collaboration 2024–2025

Multimodal / Multiview Learning
applied to real problems

Collaborative research between PUCPR, UFPR, UFPE, Université de Rouen and Universidad de los Andes, with an emphasis on pattern recognition, machine learning and computer vision.

View meetings Learn more

Multimodal / Multiview Learning
applied to real problems


    Abstract: We perceive the world around us in a multimodal way. Consequently, multimodal systems have received much attention in Artificial Intelligence (AI). The challenges in such an approach go far beyond the simple fusion of modalities. In practice, it is relevant to consider: a) the modalities to be used, b) the representation of each modality, c) the way modalities interact, and d) how learning accounts for different modalities (individually or simultaneously). This proposal aims to enhance our understanding of the strategies above and develop novel multimodal and multiview predictive models, investigating alternatives for representing and combining different modalities in Pattern Recognition and Machine Learning. The project specifically focuses on researching and developing solutions using multimodal prediction models, emphasizing the architecture of these models, new strategies for generating representations for each modal (views), and new strategies for the selection/fusion of views and modalities considering possible intermodal relationships. Additionally, the project comprises the investigation of strategies to improve the training process of multiple modalities. Each stage of the working plan has the potential to result in publications in qualified international scientific journals and conferences. The project's social contribution relates to human resource formation and the potential for generating technological innovations that can positively impact society in terms of emotion analysis to help in the diagnosis of depression, medical image analysis, document image retrieval in digital libraries, and car safety (driver assistance systems). Ultimately, this proposal consolidates scientific exchange and collaboration of research teams from France, Brazil, and Chile, started in SticAmsud 2014.
  • Pattern Recognition and Computer Vision
  • Multimodal and Multiview Machine Learning
  • Internacional Cooperation and Training in Machine Learning

Missions

Kick-off meeting»

Follow the records and materials of each research mission.

Mission 1
2024

Mission 1 (Rouen)

Goal alignment, task split, and experimental pipeline definition.
Mission 2
2024

Mission 2 (Recife)

Preliminary results, protocol adjustments, and metric definition.
Mission 3
2025

Mission 3 (Rouen)

Dataset integration and cross‑modality comparisons.
Mission 4
2025

Mission 4 (Chile)

Summary of progress, publication planning, and next steps.

Gallery

Highlights and visual records of the project.

Institutions

Partner universities and research centers.

Researchers

Project coordination and team members.

Prof. Alceu de Souza Britto Jr (Coordinator, PUCPR/Brazil)

Prof. Laurent Heutte (French Coordinator, Université de Rouen/France)

Prof. George D. C. Cavalcanti (Associate Coordinator, UFPE/Brazil)

Prof. Luiz Eduardo S. Oliveira (Associate Coordinador, UFPR/Brazil)

Prof. Simon Bernand (Université de Rouen/France)

Prof. Stéphane Nicolas (Université de Rouen/France)

Prof. Jean Paul Barddal (PUCPR/Brazil)

Prof. André Gustavo Hochuli (PUCPR/Brazil)

Prof. Jose M. Saavedra (Chilean Coordinator, Universidad de los Andes/Chile)

Carlos Magno (PhD student, PUCPR/Brazil)

Caio S. Dias (PhD student, PUCPR/Brazil)

Luciana Menon (PhD student, PUCPR/Brazil)

Daniel Vriessman (PhD student, PUCPR/Brazil)

Cristiano Garcia (PhD student, PUCPR/Brazil)

Everton Baro (PhD student, UFPR/Brazil)

Leandro Ensina (PhD student, UFPR/Brazil)

Francimaria R. S. Nascimento (PhD student, UFPE/Brazil)

Keila Santos (PhD student, UFPE/Brazil)

Livia Alexandre (PhD student, UFPE/Brazil)

Lucas Amorim (PhD student, UFPE/Brazil)

Cynthia Moreira Maia (PhD student, UFPE/Brazil)

Joseph Assaker (PhD student, Université de Rouen/France)

Coordinators

France, Brazil and Chile.

  • Coordination (France): Prof. Laurent Heutte ([email protected]), Laboratoire d'Informatique, du Traitement de l'Information et des Systèmes (LITIS, UR, France). Visit LITIS Lab
  • Coordination (Brazil): Prof. Alceu de S. Britto Jr. ([email protected]), Postgraduate Program in Computer Science (PPGIa, PUCPR, Brazil). Visit PPGIa
  • Coordination (Chile): Prof. Jose M. Saavedra ([email protected]), Faculty of Engineering and Applied Sciences (Universidad de Los Andes, Chile). Visit UAndes
    • © 2024–2025 MMLARP. All rights reserved.