Research Overview
My research focuses on the synergy between human and machine intelligence, particularly how insights from computational neuroscience and AI can benefit each other.
Research Interests:
- Aligning AI/ML models to human understanding
- Multimodal self-supervised learning
- Vision foundation model
Aligning AI models to human understanding
My research explores how to align AI model representations with human understanding, focusing on vision and language. By using self-supervised learning with multimodal data, we’ve made language models’ representations more closely resemble human thought processes [NeurIPS21, CRCNS22]. On the vision front, I’ve concentrated on visual attention to bring the representations of vision models in line with human perception [NeurIPS23, NECO24, COSYNE23, CRCNS22]. These approaches have proven effective in making AI develop representations that closely mirror human understanding, enhancing both the interpretability and safety of vision and language models.
Do machines see the world in a way that we do? Check my recent paper at NeurIPS-23.
Multimodal Vision Foundation Model (In Progress)
In medical imaging, I’ve spearheaded the creation of a multimodal vision foundation model [ISMRM23, NeuroImg19] designed to unlock the potential of medical datasets. This AI model, trained on large-scale datasets, is adept at identifying key features for a range of applications, including disease diagnosis and personalized treatment development. This approach significantly enhances medical data utility, offering new avenues for patient care and healthcare innovation.
Improving Efficiency and Security of AI Models
I also explore how the brain’s data-efficient learning [IJCNN17, NECO18, TSMC18, IROS17abs], notably through predictive coding, informs AI development, enhancing computer vision and robotics by predicting future sensory inputs to reduce prediction errors. My studies demonstrate significant improvements in deep neural network training efficiency through this method.