DecisionRobust benchmark tests JEV-style decision models against text and image attacks
A new arXiv paper introduces DecisionRobust, a benchmark for systematically evaluating how robust JEV-style decision models are to textual and visual inputs. It covers practical decision-making scenarios with semantics-preserving text perturbations, image perturbations and prompt injection in both modalities. The authors evaluated 30 model configurations on text tasks and 17 on vision tasks, including 24 and 12 held-out models respectively; they report that attacks expose vulnerabilities beyond the models used for construction and selection, that optimized image perturbations transfer weakly to many targets, and that resistance to input perturbations does not necessarily imply resistance to prompt injection.
As decision models such as JEV, which map instructions and observations directly to choices, judgments and scores, are increasingly used for automated decision-making, the benchmark isolates reliability under adversarial manipulation and finds that label-consistency checks substantially reduce measured attack success.
Computer Science > Computation and Language
arXiv:2610.19118 (cs)
[Submitted on 13 Oct 2026]
Title:DecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and Vision
Authors:Yifan Lan, Zhongye Liu, Yurui Chang, Hanyu Wang, Yongkang Du, Lu Lin, Jinghui Chen
View a PDF of the paper titled DecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and Vision, by Yifan Lan and 6 other authors