← All publications

Validation-Selected Linear Interpolation Between Qwen2.5-0.5B Base and Instruct: Does It Beat Both Parents on ARC, HellaSwag and WinoGrande?

Dheiver Francisco Santos, PhD · Preprint, Figshare, 15 November 2026 · CC BY 4.0 · doi:10.6084/m9.figshare.35013979 · ORCID 0000-0002-8599-9436

Abstract

Interpolating the weights of a pretrained model and its fine-tuned sibling has beaten both endpoints in some vision settings. We ask whether the same holds for a small language model. We interpolate linearly between Qwen2.5-0.5B and Qwen2.5-0.5B-Instruct on eleven coefficients, choose t^* on 300 validation items per task, and report it on 3,967 disjoint test items from ARC-Easy, ARC-Challenge, HellaSwag and WinoGrande. The pre-registered criterion was a paired 95% bootstrap interval for the gain over the better original model lying entirely above zero. It was not met. The selected t^*=0.7 reaches 50.90% test accuracy against 50.25% for the base model, a difference of +0.65 pp (95% CI [-0.23,+1.52], p=0.149): a tie. The Spearman rank correlation between validation and test accuracy over the eleven coefficients is ρ=-0.03 (p=0.93), so the validation split carried no detectable information about the test ranking. By this selection procedure and this test set, gains above about 1.5 pp are excluded; smaller gains are neither confirmed nor excluded. One model pair and four benchmarks limit generality.

Download the PDF · Figshare record and reproducibility package

Cite as: D. F. Santos, “Validation-Selected Linear Interpolation Between Qwen2.5-0.5B Base and Instruct: Does It Beat Both Parents on ARC, HellaSwag and WinoGrande?,” Figshare, 2026, doi:10.6084/m9.figshare.35013979.