Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

SQAM: Q-learning with scalar adjoint matching for flow policies

AI summary

An arXiv preprint proposes Q-Learning with Scalar Adjoint Matching (SQAM). The authors observe that the batch-averaged velocity Jacobian of pretrained flow policies concentrates on its diagonal, and use this to derive a closed-form scalar adjoint that scales the value gradient at the final action by flow time, removing per-step vector–Jacobian products. SQAM pairs this with a value penalty at policy-generated actions. It beats the strongest baseline in each of the four hardest OGBench domains by 18 to 35 percentage points, and when fine-tuning a vision-language-action policy on a real bimanual robot it improves over supervised fine-tuning on all three tasks.

In-depth insight

If the simplification holds, it could cut the compute cost of fine-tuning flow policies and robot policies with off-policy reinforcement learning.

Computer Science > Machine Learning

arXiv:2610.10437 (cs)

[Submitted on 7 Oct 2026]

Title:Q-Learning with Scalar Adjoint Matching

Authors:Yonghoon Dong, Minsung Yoon, Jaehyuk Kim, Jungwoo Park, Changyeon Kim, Jinwoo Shin

View a PDF of the paper titled Q-Learning with Scalar Adjoint Matching, by Yonghoon Dong and 5 other authors

#OGBench

Original · 3 min readRead the original
Group chat

No comments yet. Start the conversation.