Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

K-FAC

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 2

  1. CurvDelta targets safer LLM adaptation and selective unlearning

    A research paper submitted by Zeguan Xiao and five coauthors introduces CurvDelta, a framework that embeds capability-preservation constraints into model parameter updates. CurvDelta uses low-curvature geometry, K-FAC and factorized projection, while CurvLoRA provides a low-rank variant; the authors report competitive task performance with better general-capability retention on mathematical reasoning, code generation, TOFU and MUSE.

    Hugging Face · Papers · 🔥 0
  2. FOAM Halves SOAP Optimizer State and Matches Reported Baselines

    A paper by Jaemyung Yu, Byeongho Heo, Sangdoo Yun, and Dongyoon Han introduces FOAM, a lightweight SOAP variant designed to reduce optimizer-state memory and eigendecomposition overhead. FOAM uses input-only preconditioning and replaces SOAP’s dense rotated second moment with a factored Adam-moment estimator based on row-by-column statistics. The authors report that, in LLM pre-training experiments from 124M to 760M parameters, FOAM matches strong SOAP and Muon baselines in validation loss, uses half of SOAP’s optimizer state, and runs at Muon-class wall-clock time. The supplied report presents these results as the authors’ findings and does not independently verify them.

    Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my K-FAC experienceAsk about K-FAC

Nobody has shared their experience with K-FAC yet.