← All publications

The Language-Modeling Cost Along the Base-to-Instruct Weight Interpolation Path of Qwen2.5-0.5B: WikiText-2 NLL versus ARC, HellaSwag and WinoGrande

Dheiver Francisco Santos, PhD · Preprint, Figshare, 15 November 2026 · CC BY 4.0 · doi:10.6084/m9.figshare.35014015 · ORCID 0000-0002-8599-9436

Abstract

Linear interpolation between a base language model and its instruction-tuned version, θ(t)=(1-t),θ_base+t,θ_inst, is often tuned for task accuracy alone. We measure what the same path does to language-modeling quality. For Qwen2.5-0.5B and its Instruct version, we evaluate 11 coefficients (t=0,0.1,…,1) on a macro-average of four multiple-choice benchmarks (3967 test items) and on WikiText-2 negative log-likelihood (NLL; 100 test blocks). All selection uses a disjoint validation split. The pre-registered hypothesis was that the accuracy-optimal t^* differs from the NLL-optimal one. It holds only literally: validation picks t^*_NLL=0.1 and t^*_acc=0.7, but t^*_NLL=0.1 is not distinguishable from the base (the interval contains zero) and t^*_acc=0.7 rests on about one net validation item (0.0008). Test NLL rises from t≈0.2 onward, reaching +0.0886 nats per token [+0.0831, +0.0939] at t=1. Accuracy shows no detectable difference from the base (+0.65 pp, 95% CI [-0.23,+1.52]). Evaluation in bfloat16 agrees with fp32 to within 0.003 nats. The study covers one model pair, one scale and one path.

Download the PDF · Figshare record and reproducibility package

Cite as: D. F. Santos, “The Language-Modeling Cost Along the Base-to-Instruct Weight Interpolation Path of Qwen2.5-0.5B: WikiText-2 NLL versus ARC, HellaSwag and WinoGrande,” Figshare, 2026, doi:10.6084/m9.figshare.35014015.