Developer releases open-source Deepthink agent-scaling system
Ryoiki-Tokuiten announced Deepthink, an open-source system for applying test-time scaling with agent swarms to difficult problems. The developer tested it only with Gemma4-35B and Qwen-3.8-27B, reporting that it used about 200 million tokens per problem and raised scores from 14% to 65% on 15 BrokenArxiv problems and from 9% to 52% on eight DeepSWE problems. These results were not independently reproduced, and the developer said runs could take from 50 minutes to 30 hours depending on configured depth and inference speed.
Deepthink provides an open-source implementation of test-time scaling with agent swarms, but its reported accuracy gains, general usability and compute requirements have not been independently validated.
| I have been working on test-time scaling for quite some time now. Posted some of my previous work on this sub-reddit before. Recently, I made significant simplifications to the core system-design and it finally felt like now anybody can understand and directly use this on their hardest problems. Just on a side-note though, depending on the depth you set & your LLM's inference speed, it can take from 50 Minutes to 30 hours to give you back the results. For benchmarks I couldn't test this on frontier models or even large open models because it will be way too expensive. I directly only tested this on Gemma4-35B & Qwen-3.8-27B. They spent around 200M tokens / problem and there scores went from 14% to 65% on BrokenArxiv (Matharena) 15 problems or from 9% to 52% on 8 DeepSWE problems. I know that's almost unbelievable, but you can try and see that yourself. Repo Link: https://github.com/ryoiki-tokuiten/Deepthink submitted by /u/Ryoiki-Tokuiten [link] [comments] |