---
title: Efficient Adversarial Training in LLMs with Continuous Attacks
url: https://www.ml-quant.com/papers/arxiv/2405.15589/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2405.15589
source_url: https://arxiv.org/abs/2405.15589
featured: 2024-11-06
citations: 148
topic: LLMs & Text
---


# Efficient Adversarial Training in LLMs with Continuous Attacks

CAdvUL, a new adversarial training algorithm, enhances the resilience of large language models against adversarial attacks by efficiently calculating attacks in the continuous embedding space.

- Source: https://arxiv.org/abs/2405.15589
- Identifier: arXiv:2405.15589
- Released: 2024-05-24
- First featured: Quant Letter No. 73 (2024-11-06): https://www.ml-quant.com/issues/2024-11-06/
- Citations (Semantic Scholar): 148
- Published in: Neural Information Processing Systems
- Topic: LLMs & Text

## Related

- [SliceGPT: Compress Large Language Models by Deleting Rows and Columns](https://www.ml-quant.com/papers/arxiv/2401.15024/): Compressing Language Models: The paper introduces SliceGPT, a post-training sparsification scheme for large language models that reduces the network's embedding dimension, maintains high performance, reduces inference computation, and reveals computational invariance in transformer networks.
- [Gecko: Versatile Text Embeddings Distilled from Large Language Models](https://www.ml-quant.com/papers/arxiv/2403.20327/): Compact Text Embeddings: Gecko is a new text embedding model that improves knowledge extraction from large language models, surpassing other models in the Massive Text Embedding Benchmark.
- [Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free](https://www.ml-quant.com/papers/arxiv/2410.10814/): The research shows that Mixture-of-Experts Large Language Models can be effective embedding models without finetuning, and suggests a combination of routing weights and hidden state for better performance.
- [CoLLEGe: Concept Embedding Generation for Large Language Models](https://www.ml-quant.com/papers/arxiv/2403.15362/): Embedding Generation: CoLLEGe, a new meta-learning framework, is presented which enhances the ability of language models to learn new concepts quickly using a few example sentences or definitions.
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://www.ml-quant.com/papers/arxiv/2408.03314/): The research investigates enhancing Large Language Models' (LLMs) performance using more test-time computation, suggesting a compute-optimal scaling strategy based on prompt difficulty.
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://www.ml-quant.com/papers/arxiv/2412.05271/): The paper presents InternVL 2.5, a sophisticated multimodal large language model that performs well on various benchmarks, exceeding 70% on the MMMU benchmark.