Sony patent would use AI to predict a game's next frame before GPU rendering completes
Sony has filed a patent for an AI system that predicts and displays a game's next frame before the graphics hardware finishes rendering it, aiming to cut stuttering without the extra latency of some frame-generation methods. TechSpot reports that instead of waiting for two completed frames to interpolate between, Sony's system uses the most recently completed frame, optionally plus motion vectors and recent controller input, to generate the predicted frame. The design uses a jointly trained encoder that converts a rendered frame into an embedding and a decoder that generates the predicted image, which is sent to the display buffer while the hardware renders the next frame. The filing includes no performance measurements, so its effect on latency and visual quality is unproven; Sony also describes a non-AI alternative that shifts pixel blocks according to their motion.
The approach targets the latency penalty of interpolation-based frame generation, and the reports say it could help games hold a steadier frame delivery rate when rendering slows down — though with no benchmarks, the practical result remains untested.
Serving tech enthusiasts for over 25 years. TechSpot means tech analysis and advice you can trust.
Forward-looking: Sony has filed a patent for an AI system that predicts a game's next frame before the graphics hardware finishes rendering it, aiming to reduce stuttering without introducing the additional latency associated with some frame-generation methods. The system uses the most recently completed frame and, optionally, recent controller input to generate a predicted frame that can be displayed while the hardware renders the next one.
The approach targets a drawback of some frame-generation techniques: they require two completed frames before generating an image to insert between them. Waiting for the second frame adds latency. Sony's system would instead use an already available frame to predict the next one, rather than waiting to interpolate between two completed frames.
That difference could help games maintain a steadier frame delivery rate when rendering slows down. However, the patent filing includes no performance measurements, leaving its effects on latency and visual quality unproven.
Sony's design uses a jointly trained encoder and decoder. The encoder converts a rendered frame into an embedding – a compact numerical representation of the image. The decoder then uses that representation to generate a predicted frame.