Academic Portfolio

Research & Publications

Investigating the frontiers of intelligence through the lens of robustness, efficient adaptation, and multi-modal synthesis.

analytics Academic Impact
2,876 Citations
psychology Scholarly Depth
H-Index 17
library_books Volume
38 Publications
memory

LLM POST-TRAINING

Focus Area

Optimization strategies for large language models, focusing on Parameter-Efficient Fine-Tuning (PEFT) and continuous domain adaptation without catastrophic forgetting.

PEFT LORA TRANSFORMERS

SMT: Fine-tuning large language models with sparse matrices

TL; DR: Instead of low-rank updates, selecting task-relevant sub-matrices enables PEFT to outperform LoRA and better match full fine-tuning.

H He, JB Li, X Jiang, H Miller.
COLM 2025 (In submission)

Finetuning MoE LLMs with Condenser Experts

TL; DR: We stabilize MoE fine-tuning by eliminating auxiliary losses and preserving rare expert knowledge, achieving stronger downstream performance.

security

Robust AI

Focus Area

Ensuring model safety and consistency through adversarial training, uncertainty quantification, and structural bias mitigation in high-stakes environments.

ADVERSARIAL UNCERTAINTY RELIABILITY
ICASSP 2022 Best Student Paper

On adversarial robustness of large-scale audio visual learning

JB Li, S Qu, X Li, PYB Huang, F Metze.

Adversarial camera stickers: A physical camera-based attack on deep learning systems

JB Li, FR Schmidt, JZ Kolter.
NEURIPS 2019

Adversarial music: Real world audio adversary against wake-word detection system

JB Li, S Qu, X Li, J Szurley, JZ Kolter, F Metze.
graphic_eq

Audio & Multimodal

Focus Area

Cross-modal representation learning and generative audio synthesis, exploring the intersection of visual semantics and spatial audio reconstruction.

SPECTROGRAMS CROSS-MODAL SPATIAL
ICML 2023 Workshop + ICASSP 2024

Audio-journey: Efficient visual+ llm-aided audio encodec diffusion

JB Li, JS Michaels, L Yao, L Yu, Z Wood-Doughty, F Metze.

Masked autoencoders that listen

PY Huang, H Xu, JB Li, A Baevski, M Auli, W Galuba, F Metze, ...

Towards Zero-shot Learning for Automatic Phonemic Transcription

X Li, S Dalmia, DR Mortensen, JB Li, AW Black, F Metze.
ICMR 2018 Best Paper

Joint embeddings with multimodal cues for video-text retrieval

NC Mithun, JB Li, F Metze, AK Roy-Chowdhury.