OpenEBFM
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- OpenEBFM releases code, weights and training stack
The OpenEBFM authors released the full training and evaluation stack, pretrained checkpoints, training recipes and an EBFM-based chatbot. In their experiments, EBFMs achieved lower validation bits per byte than matched GPT baselines during data scaling, with the gap generally increasing as the training-token budget grew. From roughly 268 million to 1 billion parameters, EBFMs improved faster than corresponding GPT baselines during parameter scaling. On DCLM, the largest EBFM reached an average zero-shot question-answering accuracy of 44.71%, compared with 39.79% for the corresponding GPT baseline in the authors’ setup. The authors cautioned that these self-reported results do not establish that energy-based models are universally better than autoregressive models, and noted that the experiments remain at a scale of about 1 billion parameters.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my OpenEBFM experienceAsk about OpenEBFM
Nobody has shared their experience with OpenEBFM yet.