Ben Affleck on machine learning and Python in VFX
A quote posted on Simon Willison's blog has Ben Affleck describing his long interest in computers and how visual-effects workflows have for years included machine learning. He points to convolutional neural networks — precursors to the transformer — operating on tensors (the numeric encoding of frame, batch and RGB pixel values) to do edge detection and feature extraction, identifying things like a window ledge so a green screen can be pulled and replaced. He adds that he can write "pretty shitty Python scripts."
Why it matters: It is a practitioner's anecdote with no new model, dataset or product information, so it has little bearing on AI practitioners' day-to-day decisions.
7th October 2026
I've always been kind of into computers since I was young. And then when film started to move from analog film to digital, I became more interested in that aspect of it. And the visual effects workflow for many years has included machine learning.
So I can write like pretty shitty Python scripts and stuff like that because with convolutional neural networks, which were the sort of precursors to what the transformer can do, which is just much more computation simultaneously, you would do things like look at what's called a tensor, which is just the numerical translation of a visual image in numbers — like the batch number, the frame number, the red, green, and blue values of each pixel in each frame.
And a tensor, you use a convolutional neural network to identify patterns in that that would reveal what's called edge detection or feature extraction, which is just identifying patterns enough to know like this is where the window ledge is, so we can more easily take the green screen image out and replace it with something.
— Ben Affleck, Pythonista