Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Developer releases open-source Deepthink agent-scaling system

AI summary

Ryoiki-Tokuiten announced Deepthink, an open-source system for applying test-time scaling with agent swarms to difficult problems. The developer tested it only with Gemma4-35B and Qwen-3.8-27B, reporting that it used about 200 million tokens per problem and raised scores from 14% to 65% on 15 BrokenArxiv problems and from 9% to 52% on eight DeepSWE problems. These results were not independently reproduced, and the developer said runs could take from 50 minutes to 30 hours depending on configured depth and inference speed.

In-depth insight

Deepthink provides an open-source implementation of test-time scaling with agent swarms, but its reported accuracy gains, general usability and compute requirements have not been independently validated.

I have been working on test-time scaling for quite some time now. Posted some of my previous work on this sub-reddit before. Recently, I made significant simplifications to the core system-design and it finally felt like now anybody can understand and directly use this on their hardest problems. Just on a side-note though, depending on the depth you set & your LLM's inference speed, it can take from 50 Minutes to 30 hours to give you back the results. For benchmarks I couldn't test this on frontier models or even large open models because it will be way too expensive. I directly only tested this on Gemma4-35B & Qwen-3.8-27B. They spent around 200M tokens / problem and there scores went from 14% to 65% on BrokenArxiv (Matharena) 15 problems or from 9% to 52% on 8 DeepSWE problems. I know that's almost unbelievable, but you can try and see that yourself. Repo Link: https://github.com/ryoiki-tokuiten/Deepthink submitted by /u/Ryoiki-Tokuiten [link] [comments]

#Deepthink#OpenAI#Gemma4-35B#Qwen-3.8-27B

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.