Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Edge0 Opens 145B and 120B On-Device Models

AI summary

Edge0 open-sourced the Edge0-145B and Edge0-120B models, along with a unified inference framework and deployment tools. Edge0 describes Edge0-145B as a sparse model with 145 billion total parameters and about 10 billion active parameters. The company reports that it runs locally on compatible laptops with peak memory of 29GB and decoding above 80 tokens per second. A commentary report described hands-on tests of local 3D modeling, game and presentation workflows and noted that benchmark comparisons relied on Edge0-published results.

In-depth insight

The release provides model weights and a local deployment stack for coding, reasoning and agent workflows, giving developers a way to evaluate, fine-tune or integrate models without relying entirely on cloud inference. It may also reduce cloud API fees and network dependence while keeping data on the computer, although suitable hardware is required and the reported performance and benchmark comparisons were not independently established.

Today, we're open-sourcing Edge0-145B and Edge0-120B, together with our unified inference framework and deployment tools. Edge0-145B is the world's first open-source on-device AI model with 145 billion parameters. It brings coding, reasoning, and agent capabilities that rival frontier cloud models to your own computer.

On-device AI has long forced a tradeoff between model capability and what the hardware can run. Larger models demand more memory, bandwidth, and power; smaller models can struggle with complex work. Moving that work to the cloud adds model-provider fees, data-handling constraints, and a dependence on connectivity.

This release addresses those constraints in four ways:

The constraintWhat Edge0-145B makes possible
Large models are difficult to run on personal hardware.A 145B model runs locally on a compatible laptop, with reported peak memory of 29GB and decoding above 80 tokens/s.
Small on-device models can fall short on demanding tasks.Coding, reasoning, and instruction-following results are competitive with specific frontier cloud models, as the comparisons below show.
A useful assistant needs to finish work, not just answer questions.The model plans, writes and runs code, debugs errors, and checks results visually for projects such as 3D scenes, games, hardware programs, and PCB designs.
Cloud workflows bring recurring fees, sensitive-data restrictions, and network dependence.Local inference incurs no corresponding cloud-model token or API fees; data can stay on the computer, and core capabilities keep working offline. Hardware and electricity still cost money.

#Edge0#Edge0-145B#Edge0-120B#Edge0 App

Original · 16 min readRead the original
Group chat

No comments yet. Start the conversation.