Machine learningLLMs & Text

Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models

Synthesis Step by Step (S3) is a new data synthesis framework designed to minimize the distribution gap between synthesized and real task data, improving the performance of small models trained on the synthesized dataset.