Meta Crawlers Overload a Small Creator’s Websites
Meta web crawlers generated more than 550,000 visits to illustrator Chelsea Paulus’s four websites, exhausting shared hosting resources and leaving the sites unstable, according to IT之家 citing Business Insider. SiteGround alerted Paulus on October 24 that 80% of her allocated CPU was in use; traffic was associated with Meta External Agent and later Meta Web Indexer. Paulus said she does not believe Meta intentionally launched an attack and called for accessible tools that small businesses can use to block crawlers.
The reported case shows how Meta crawler traffic can directly strain shared hosting for a small website operator, while Meta points site owners to robots.txt and crawler configuration tools as ways to control interactions.
IT之家 11 月 3 日消息,圣地亚哥一名创作者原本希望有更多人访问自己的网站,结果她迎来的不是访客,而是 Meta 爬虫带来的 55 万余次访问,如今她只能手忙脚乱地维持网站正常运转。

儿童绘本作家、插画师切尔西 · 保卢斯(Chelsea Paulus)向 Business Insider 表示,这台自动访问程序耗尽了网站主机资源,她不得不和服务商一起排查故障。
“我从来没有遇到过这种情况,”保卢斯说,“哪怕我懂一些技术,这周也过得心力交瘁。”
保卢斯经营这份事业已有十年,去年转为全职创作者。此前她有科技和网页设计从业背景,所以流量异常暴涨时,她很快察觉到不对劲。
“我不认为这是 Meta 蓄意发起的恶意攻击,”保卢斯说,“但小微企业主需要知道这类风险,并且要有可用工具来阻挡爬虫。”
#Meta#Chelsea Paulus#Meta External Agent#Meta Web Indexer#SiteGround#WordFence#Cloudflare