疑似Roblox爬虫抓取145.5GB 3D资产
Open3DLab称,一个自称属于Roblox研究的爬虫在34小时内发起26,429次请求,读取了18,926个项目页面。随后该爬虫从Open3DLab、SFMLab和SmutBase的712个项目中下载1,027个文件,共145.5GB,其中包括成人内容,且无视robots.txt与创作者许可。Roblox尚未确认该爬虫由其运营,也没有证据显示这些文件被用于训练或评估模型。
这份基于服务器日志的记录揭示了一起涉及3D资产、许可限制和成人内容网站的大规模AI相关数据收集事件,但Roblox的归属与数据最终用途仍未得到确认。
On 15 and 16 October 2026, a crawler that said it belonged to Roblox walked through almost every project on Open3DLab, SFMLab and SmutBase, and proceeded to download complete copies of 1,027 files from 712 projects. 145.5 GB of our creators' work, downloaded for what we assume are AI training purposes.
This post briefly describes what happened, what mitigations were taken, and what this signals for the community at large. A more detailed writeup outlining what we saw in our logs will be posted at a later time.
What happened
On Sunday the 18th of October, we noticed a crawler making a large number of requests to our sites and download infrastructure. The crawler sent the same user agent on every request, trellis2-data-sourcing/1.0 (3D foundation model research; [redacted]@roblox.com). It came from a single Amazon Web Services (AWS) address. It had read the main page of 18,926 projects, which is practically our entire catalogue, in 26,429 requests over a span of 34 hours.
After pulling the project pages, it downloaded complete copies of 1,027 files from 712 projects, 145.5 GB in all, before suspending operations during the weekend. The downloads include 364 files (54.8 GB) from 224 projects on SmutBase, which hosts adult content only. With the SFMLab projects marked adult, 345 of the 712 projects (62.6 GB) are adult content.
#Roblox#Open3DLab#SFMLab#SmutBase#Amazon Web Services#Roblox Cube