Dietrich Klakow

Probing Chemical Language Models: Effects of Pre-training and Fine-tuning

Anna Karnysheva, Dietrich Klakow, Ji-Ung Lee

OpenBibleTTS: Large-Scale Speech Resources and TTS Models for Low-Resource Languages

David Guzmán, Luel Hagos Beyene, Jesujoba Oluwadara Alabi, et al.

On Optimizing Multimodal Jailbreaks for Spoken Language Models

Aravind Krishnan, Karolina Stańczak, Dietrich Klakow

This paper demonstrates that Spoken Language Models (which process both speech and text) can be effectively attacked by simultaneously manipulating both audio and text inputs together, rather than just one modality alone. The researchers developed JAMA, a technique that optimizes adversarial perturbations across both modalities at the same time, and showed it achieves 1.5 to 10 times higher attack success rates than previous single-modality attacks. The findings highlight that protecting these AI systems requires considering both input types together, not separately.