Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Reddit user questions OKLS step efficiency versus Muon

AI summary

A Reddit post asks why OKLS, described as a zero-staleness implementation of KL-Shampoo, may be outperformed by Muon in step efficiency. The submitter claims that SOAP is more step-efficient than Shampoo, KL-Shampoo is more step-efficient than SOAP, and OKLS only surpasses Muon in parameter efficiency during very long training runs. The supplied post does not include benchmark measurements, implementation details, or responses that resolve these comparisons.

SOAP is more step efficeint than Shampoo, and KL-Shampoo is more step efficient than SOAP.

Then how is OKLS getting outperformed by Muon in terms of step efficiency where it's a zero-staleness implementation of KL-Shampoo?

It only gets ahead of Muon in terms of parameter efficiency in very long training runs.

#OKLS#Muon#KL-Shampoo#SOAP#Shampoo

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.