---
title: Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
url: https://www.ml-quant.com/papers/arxiv/2407.10930/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: arXiv:2407.10930
source_url: https://arxiv.org/abs/2407.10930
featured: 2024-07-17
citations: 59
topic: LLMs & Text
---


# Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together

The article explores a method to enhance Natural Language Processing systems by simultaneously optimizing language model weights and prompting strategies, leading to significant improvements in tasks like multi-hop QA and mathematical reasoning.

- Source: https://arxiv.org/abs/2407.10930
- Identifier: arXiv:2407.10930
- Released: 2024-07-15
- First featured: Quant Letter No. 57 (2024-07-17): https://www.ml-quant.com/issues/2024-07-17/
- Citations (Semantic Scholar): 59
- Published in: Conference on Empirical Methods in Natural Language Processing
- Topic: LLMs & Text

## Related

- [Llemma: An Open Language Model For Mathematics](https://www.ml-quant.com/papers/arxiv/2310.10631/): Open Math Language Model: The article discusses Llemma, a superior language model for mathematics that can prove theorems without additional fine-tuning.
- [Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://www.ml-quant.com/papers/arxiv/2405.05904/): Research shows that large language models have difficulty acquiring new factual knowledge through fine-tuning, learning new information slower than consistent knowledge, and are more likely to hallucinate, indicating the risks of introducing new facts through fine-tuning.
- [TableLlama: Towards Open Large Generalist Models for Tables](https://www.ml-quant.com/papers/arxiv/2311.09206/): The paper presents TableLlama, an open-source large language model fine-tuned for table-based tasks, and introduces a new dataset, TableInstruct, which improves the performance and generalizability of these models.
- [Designing Heterogeneous LLM Agents for Financial Sentiment Analysis](https://www.ml-quant.com/papers/arxiv/2401.05799/): A study suggests using large language models without fine-tuning for financial sentiment analysis, offering a design framework that enhances accuracy.
- [Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models](https://www.ml-quant.com/papers/arxiv/2306.12659/): A new approach improves financial sentiment analysis by addressing limitations of language models.
- [AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation](https://www.ml-quant.com/papers/arxiv/2408.00764/): Enhancing LLM Planning: The study improves the planning abilities of Large Language Models (LLMs) using instruction tuning and a framework called AgentGen, which generates diverse environments and planning tasks.