DecisionRobust

Headlines · 1

1. Oct 13
   DecisionRobust benchmark tests JEV-style decision models against text and image attacks

   A new arXiv paper introduces DecisionRobust, a benchmark for systematically evaluating how robust JEV-style decision models are to textual and visual inputs. It covers practical decision-making scenarios with semantics-preserving text perturbations, image perturbations and prompt injection in both modalities. The authors evaluated 30 model configurations on text tasks and 17 on vision tasks, including 24 and 12 held-out models respectively; they report that attacks expose vulnerabilities beyond the models used for construction and selection, that optimized image perturbations transfer weakly to many targets, and that resistance to input perturbations does not necessarily imply resistance to prompt injection.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my DecisionRobust experienceAsk about DecisionRobust