In a brand new AI research, a team of MIT and Harvard University researchers has introduced a groundbreaking framework called “Follow Anything” (FAn). The system addresses the restrictions of current object-following robotic systems and presents an progressive solution for real-time, open-set object tracking and following.

The first shortcomings of existing robotic object-following systems are a constrained ability to accommodate latest objects as a consequence of a hard and fast set of recognized categories and a scarcity of user-friendliness in specifying goal objects. The brand new FAn system tackles these issues by presenting an open-set approach that may seamlessly detect, segment, track, and follow a big selection of things while adapting to novel objects through text, images, or click queries.

The core features of the proposed FAn system may be summarized as follows:

Open-Set Multimodal Approach: FAn introduces a novel methodology that facilitates real-time detection, segmentation, tracking, and following of any object inside a given environment, no matter its category.

Unified Deployment: The system is designed for straightforward deployment on robotic platforms, specializing in micro aerial vehicles, enabling efficient integration into practical applications.

Robustness: The system incorporates re-detection mechanisms to handle scenarios where tracked objects are occluded or temporarily lost in the course of the tracking process.

The basic objective of the fan system is to empower robotic systems equipped with onboard cameras to discover and track objects of interest. This involves ensuring the item stays inside the camera’s field of view because the robot moves.

FAn leverages state-of-the-art Vision Transformer (ViT) models to attain this objective. These models are optimized for real-time processing and merged right into a cohesive system. The researchers exploit the strengths of assorted models, corresponding to the Segment Anything Model (SAM) for segmentation, DINO and CLIP for learning visual concepts from natural language, and a light-weight detection and semantic segmentation scheme. Moreover, real-time tracking is facilitated using the (Seg)AOT and SiamMask models. A lightweight visual serving controller can be introduced to manipulate the object-following process.

The researchers conducted comprehensive experiments to guage FAn’s performance across diverse objects in zero-shot detection, tracking, and following scenarios. The outcomes demonstrated the system’s seamless and efficient capability to follow objects of interest in real-time.

In conclusion, the FAn framework represents an encompassing solution for real-time object tracking and following, eliminating the restrictions of closed-set systems. Its open-set nature, multimodal compatibility, real-time processing, and flexibility to latest environments make it a big advancement in robotics. Furthermore, the team’s commitment to open-sourcing the system underscores its potential to profit a wide selection of real-world applications.

Take a look at the Paper and Github. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to hitch our 29k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the most recent AI research news, cool AI projects, and more.

Should you like our work, please follow us on Twitter