MultiLangBench

Headlines · 1

1. Oct 19
   OpenAI says GPT-6 Astra sharply improves coding in COBOL, Fortran and Visual Basic

   OpenAI published research saying its latest models are substantially better at building and migrating software in low-resource languages such as COBOL, Fortran and Visual Basic, which it says have far less training code than Python or JavaScript yet support critical infrastructure. On MultiLangBench, which asks models to build a SQLite-compatible database engine from scratch in 20 languages, OpenAI reports GPT-6 Astra reached an 87.4% average test pass rate at maximum reasoning effort versus 26.8% for GPT-5.6 Sol, with COBOL rising from 10.9% to 72.1% and cheaper GPT-6 Luna averaging 40.3%. On a separate COBOL-to-C++, Java, Kotlin, Python or Rust migration benchmark, OpenAI says GPT-6.1 Sol scored 86.6% at low reasoning effort versus 5.4% for GPT-5.6 Sol, and 97.9% at higher effort. OpenAI cautions that applying such migration results to production software still requires expert review and testing against business rules and dependencies.

   OpenAI · 🔥 0

Experience and discussion from the community

Share my MultiLangBench experienceAsk about MultiLangBench