SJTU's Cai Deng: post-deployment self-evolution is the fourth scaling axis
Cai Deng, who worked on retrieval-augmented generation, long-context and memory mechanisms at Tencent AI Lab and ByteDance Seed, argues in a DeepTech interview that after pretraining, post-training and inference-time scaling, a fourth scaling axis is "post-deployment scaling": experience and feedback accumulated from real work interactions. He is set to join Shanghai Jiao Tong University's AI school as a tenured-track associate professor in November and to found ContinuousAI Lab, saying external memory systems and long context hit inherent limits because some experience cannot be written down or retrieved in one pass, so continual learning must update model weights. He calls in-context learning the endgame learning method, with scale as the key bottleneck, and cites InfiniteICL (ACL 2025), which reportedly moved 90% of context into parameters while keeping 10%, averaging 103% of full-context performance and 117% on multi-hop reasoning, with a claim that 0.4% of a 2M-token context beat a 128K full-context baseline; Temp-LoRA (early 2024) was an earlier internalization attempt. He frames RSI and AI for AI as pre-deployment optimization of the average across deployments.
If post-deployment continual learning works, competition could shift from general model capability toward each user's and organization's own evolving model.