Follow the Gradient: A Dataset-Scale Filter That Finds the Fine-Tuning Data Eroding Model Safety

Fine-tuning a model on instruction data can quietly strip its safety, and the culprit examples look harmless on inspection — which is exactly why content filters miss them. A new paper from Datology AI and UIUC, arXiv 2610.37481, attacks the problem at the gradient level: find the training examples whose gradients point toward unsafe behavior, and remove them before fine-tuning happens.

SimpleLoRIF

The method, SimpleLoRIF, simplifies a low-rank attribution technique and restricts backpropagation to the final four transformer blocks. That combination is what makes it practical: gradient attribution stops being a small-scale analysis toy and becomes a filter that can embed datasets on the order of a million examples — the scale of a real post-training corpus.

The Emergent Misalignment Signal

Applied to emergent misalignment, the method connects insecure-code training data to the broadly harmful behaviors that surface after fine-tuning, reaching an AUC as high as 0.865 in the gradient space of misaligned models. In other words, the harmful trajectory has a measurable fingerprint in the training gradients — before the model ever exhibits it.

SafeTulu

Using the filter, the authors built SafeTulu: Tulu 3 with the 20% of examples most gradient-aligned with unsafe responses removed. Training on it reduces GCG attack success rate by up to 22% across Qwen, Llama and Olmo, beating both LLM-as-a-judge filtering and prior gradient-based approaches — while largely preserving capability, and in some cases improving it.

The authors then read the discarded data, which is arguably the more interesting artifact. They surface possibly under-studied sources of safety degradation: tool-execution failures and narrative arithmetic. Neither looks like “unsafe content” to a classifier; both quietly teach the model that shortcuts work.

Why It Matters

Safety filtering has largely been a text-level problem: judge whether an example looks bad. This work argues the signal sits one level deeper — in how an example changes the model’s weights — and that signal is cheap enough to compute at corpus scale. Both SafeTulu and a Hugging Face Space for building custom variants (adjustable removal fraction, subset-only filtering) are released, so the recipe can be applied to any fine-tuning corpus.