Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Weight-Space Programming yields localized gains with uneven reliability

AI summary

A paper shared on r/LocalLLaMA presents Weight-Space Programming (WSP), which allows reasoning agents to inspect and directly modify neural-network parameters as an alternative to gradient-based adaptation. Across 16 editing trajectories, seven benchmarks, and two model scales—Qwen3.5-0.8B and Qwen3.5-9B—the authors reported held-out accuracy gains of up to 29.45, 4.42, 36.72, and 17.11 percentage points on MATH-500, GPQA Diamond, IFEval, and SciBench, respectively. The authors also reported a 29.45-point MATH-500 gain from changing 1,023 parameters. However, results varied substantially across trajectories, and six of the 16 retained the original weights when candidate improvements could not be reliably validated.

In-depth insight

The paper reports that reasoning agents can adapt models through highly localized weight edits, while also documenting difficulties in reliably discovering effective interventions and transferring gains across tasks.

Weight-Space Programming (WSP) enables reasoning agents to directly inspect and modify neural network parameters, offering an alternative to gradient-based model adaptation. However, its capabilities, cross-task generalization, and practical limitations remain insufficiently understood. We systematically investigate WSP through 16 editing trajectories across seven benchmarks and two model scales (Qwen3.5-0.8B and Qwen3.5-9B). Our findings reveal three key insights. First, WSP can achieve substantial performance gains through highly localized interventions. On full held-out evaluations, accuracy improves by up to 29.45, 4.42, 36.72, and 17.11 percentage points on MATH-500, GPQA Diamond, IFEval, and SciBench, respectively. Notably, modifying only 1,023 parameters yields a 29.45-point improvement. Second, WSP exhibits promising but uneven cross-task generalization. A Math-edited checkpoint improves GPQA Diamond, IFEval, and SciBench by 3.03, 12.18, and 20.10 points, respectively, without further editing, while leaving SciCode performance unchanged. Third, effective editing remains challenging and inconsistent. Outcomes vary substantially across trajectories, and six of the 16 trajectories retain the original weights when candidate improvements cannot be reliably validated. Together, these results highlight WSP's potential for highly localized model adaptation while revealing important challenges in intervention discovery, reliability, and cross-task transfer. Related Thread : [Paper] Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights submitted by /u/pmttyji [link] [comments]

#Weight-Space Programming#Qwen3.5-0.8B#Qwen3.5-9B#Qwen

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.