Qin Haotong on pushing AI model compression to the limit
DeepTech interviewed Haotong Qin, an assistant professor at Hong Kong Polytechnic University, about his research on model efficiency, including cost, inference latency, and retaining capability under limited resources. Qin described BiLLM, an ICML 2024 study that compressed the average weight bit width of a pretrained language model to about 1.08 bits while retaining some language capability. He also said that lower latency in physical-system deployments can make compressed models more useful in real-world decision-making.
BiLLM demonstrates an extreme post-training quantization case, while Qin's account highlights the practical tension between model accuracy, deployment cost, inference speed, and capability retention.
在秦浩桐看来，AI 越强，值得研究的问题就越多。AI 聊天模型刚出现时，人们拿它聊天、解答困惑。后来发现它能写代码、处理任务，又开始希望它多做一点，最好能替自己把事情办完。模型的能力在长，人们的要求也在长。原来还没有人想到的用途，一旦变得可能，马上就会暴露出一批新的难题。
所以，当有人觉得 AI 已经很难再找到“无人区”，秦浩桐的判断则相反：“处处都是无人区。”在他看来，一个问题解决以后，很快就会变成基础知识，大家习以为常，容易忘记它也曾经困住过人。可模型刚刚能够触及一个新任务，和它真正能把任务做好，往往还隔着很远。
#BiLLM#The Hong Kong Polytechnic University#ETH Zurich#Beihang University#ICML