ExtractBench Pro

Headlines · 2

1. Oct 30
   LlamaIndex Extract V2.6 adds max effort

   LlamaIndex introduced Extract V2.6 with a max effort setting for the agentic_plus tier, aimed at improving data extraction from complex documents through additional computation and, when needed, higher-resolution inspection of visual content. The company reported that Value F1 on ExtractBench Pro rose from 77.7 with standard Agentic Plus to 92.9 with max effort, but the gains varied by category and were negative for charts and counting. Billing is variable and may cost more than standard agentic_plus when greater computation produces a material quality improvement.

   LlamaIndex Blog · 🔥 0
2. Oct 15
   LlamaIndex releases ExtractBench Pro for document extraction's long tail

   LlamaIndex has released ExtractBench Pro, an extension of the ExtractBench benchmark it launched two months earlier, built to cover long-tail customer documents and schemas the original 370-document set never covered. All 100 documents are synthesized by deterministic pipelines from seeds and configs, and are split into six challenges: document conventions, house vocabulary, house counting rules, drawn relations, charts inside documents and long chart sheets. LlamaIndex says it tested twelve systems — four extraction APIs, five coding agents and its three LlamaExtract tiers — and added a metric for the proportion of documents with perfect 100% Value F1; the results are vendor-run with no independent replication supplied.

   LlamaIndex Blog · 🔥 0

Experience and discussion from the community

Share my ExtractBench Pro experienceAsk about ExtractBench Pro