Small models fail the same way big models do, so this paper shows you can collect a cheap model's mistakes once and use them to make a bigger model reason better.

CritICL runs small models over math problems and saves every wrong answer with a short critique of what went wrong. When a new question arrives, the big model's prompt gets the critiques for the mistakes it is most likely to make.

That gets you the answer in 1 generation instead of 5.

Building a bank of your model family's known failure modes once, then retrieving from it per query, appears to buy accuracy that usually costs repeated generations.

This paper finds that a big model reasons better when its prompt warns it about the mistakes smaller models in its family keep making.

Instead of running your model several times and voting, you may get the same accuracy by telling it upfront which mistakes it tends to make.