Banger paper from Meta FAIR on catching distilled models.

(bookmark it)

At a 0.1% false-positive rate, their detector flags 79% of models distilled from watermarked outputs. The best token-level test flags 29%.

When a student trains on watermarked teacher traces, the watermark key leaves a specific direction in the student's weights. That direction sits apart from the normal fine-tuning directions, so it stays even when the student's text shows no token-level trace of the watermark.

WM-Basin only needs API access. It turns the suspect model's outputs into gradients on a clean probe model, removes the task directions, and checks what remains against the watermark key.

The signal holds after quantization, pruning, and RL post-training. Students range from 135M to 8.2B parameters.

https://arxiv.org/abs/2610.37266

Chat with Paper: https://academy.dair.ai/papers/watermark-basins-black-box-detection-of-distilled-llms-2610.37266