Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Academia Sinica advances visual AI with Taiwan GBA and G-YOLO

AI summary

Academia Sinica is developing Taiwan GBA for behavior recognition and G-YOLO for scene understanding, rather than building another large general-purpose model. Taiwan GBA currently recognizes 12 behaviors, including falls, fighting and knife-waving, while the team reported 80–90% accuracy on selected dataset tests and is adding depth estimation and multiview understanding to G-YOLO. The work was presented by institute director Liao Hung-Yuan and reported by iThome Taiwan.

In-depth insight

The modular approach targets capabilities that general-purpose vision-language models may lack, including interpreting actions, hazards, object relationships and real-time events on edge devices.

當AI模型開始能「看懂」照片,但是它真的理解眼前發生的事情嗎?

中研院資訊科學研究所所長廖弘源在周三一場活動中分享中研院的多模態AI研究成果,他以目前大型視覺語言模型(Vision Language Model, VLM)的發展為例,現在的大型模型已經具備相當程度的影像描述能力,例如Google DeepMind推出的Gemma系列視覺語言模型,可以將影像內容轉換成文字,但是,這類通用模型仍存在限制。

但是如果將一張小孩的照片輸入VLM後,模型給出的描述只有「Everything is normal」。這代表模型可以理解影像內容,但對於更細緻、更接近真實世界需求的判斷,例如該人物正在做什麼、是否發生危險、環境中物件之間有什麼關係,仍需要更深入的視覺理解能力。

「我們不希望台灣永遠只是買別人的大型模型」,廖弘源表示,大型模型固然重要,但若要建立自主能力,仍需要掌握自己的模型模組,才能依照不同場景組合應用。

#Academia Sinica#Taiwan GBA#G-YOLO#Universal Action Space#Video Swin Transformer#YOLOv9#StrongSORT#NTU RGB+D Action Recognition Dataset#University of Central Florida Crime Dataset#Kinetics-600#Google DeepMind#Gemma

Original · 7 min readRead the original
Group chat

No comments yet. Start the conversation.