Gunhee Kim

SiGMA: Sign-Guided Merging and Adaptation for Multimodal Continual Instruction Tuning

Keonhee Park, Gunhee Kim

Multimodal Continual Instruction Tuning (MCIT) is crucial for adapting Multimodal Large Language Models (MLLMs) to evolving a sequence of downstream tasks. Prior methods mostly utilize Mixture of Experts or expansion merge approach, primarily focusing on catastrophic forgetting,…Read more