OpenAI details GPT-Image-2.6 Sunburst safety stack
OpenAI Deployment Safety published a system card for ChatGPT Images 2.6, describing multiple image-specific safeguards built into GPT-Image-2.6 Sunburst. The system uses LLM-based policy checks before generation, a safety-focused multimodal model to inspect text and image inputs, and output moderation that blocks policy-violating images before display. OpenAI also added synchronous CSAM classification and expanded online and offline monitoring.
The system card provides a concrete account of how OpenAI is combining pre-generation refusals, multimodal input blocking and final-output moderation in a deployed image-generation product.
GPT-Image-2.6 Sunburst includes multiple layers of image-specific safety protection. Upstream, LLM-based policy checks are designed to block potentially violative image requests before image generation begins. Downstream, we use safety-focused image classifiers designed to block potentially violative input images from being fed into the final generation, combined analysis of input images and prompts to evaluate potential malicious edits, and a final step where we analyze whether the generated image violates our policies before we show the image to the user.
Below we explain each of these safety layers in a bit more detail:
- Upstream Refusals: Before a request is sent to the image generation model, we run LLM-based policy checks to evaluate whether the request violates policy. Requests deemed to violate policy are refused at this stage.
- Downstream Blocking: After a request reaches the image generation system, we use a safety reasoning model as a monitor to moderate both image inputs to the image generation model and the generated output. This model is a safety-focused multimodal model trained to reason about prompt intent and content policies.
- Input blocking: The monitor checks all text and image input provided to the image generation tool. If the monitor determines that any input violates policy, generation is blocked.
- Output blocking: The monitor also checks the final output image before it is shown to the user. If the monitor determines that the output image violates policy, it is blocked.