First-person view (FPV) drone racing is an exhilarating and rapidly growing sport where pilots control racing drones from a first-person perspective using specialized FPV goggles. The drones have powerful motors, lightweight frames, and high-quality cameras for low-latency video transmission on this sport. Pilots wear FPV goggles that provide a live video feed from the drone’s camera. This immersive experience allows them to see what the drone sees in real time.

Can we now have an autonomous mobile drone that may beat human champions within the race? The researchers of the Robotics and Perception group on the University of Zurich built a drone system called “SWIFT” that may race physical vehicles at the extent of the human world champions. Swift can fly at its physical limits while estimating its speed and placement within the circuit using sensors. 

Swift combines deep reinforcement learning (RL) in simulation with data collected from the physical world. It consists of a perception system that translates high-dimensional representation and a control policy that ingests the low-dimensional representation produced by the perception system and has control commands.

The perception system features a visual-inertial estimator and a gate detector ( a CNN that detects the racing gates). The detected gates are further used to estimate the trajectories of the drone in addition to the orientation of the drone required along the track. Swift does this evaluation using a camera-resectioning algorithm together with a map of the track. To get a more accurate drone orientation, they use the worldwide pose obtained from the gate detector combined with the visual-inertial estimator utilizing a filter. 

The control policy consists of two-layer perceptrons, which map the filter output to manage commands of the drone and maximize the perception objective by keeping the following gate within the camera’s field of view. Seeing the following gate is promising since it increases the accuracy of the pose estimation. Nevertheless, optimizing these methods purely in simulation will yield poor performance if there are discrepancies between simulation and reality. 

The differences between the simulated and real dynamics will cause the drone to decide on the mistaken trajectories, resulting in a crash. One other factor affecting the secure trajectories is a loud estimation of the drone’s state. The team mitigates these defects by collecting a small amount of knowledge in the true world and using this data to extend the simulator’s realism. They record the information using the onboard sensors with highly accurate estimates from a motion-capture system while the drone races through the track. 

Researchers say that Swift wins many of the races against each human pilot and archives the fastest race time recorded, with a lead of half a second over the perfect time clocked by a human pilot. They are saying it’s consistently faster than the human pilots on the turns and has a lower response time in taking off from the rostrum, a median of 120 ms before human pilots.  

Try the Paper. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to hitch our 29k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the newest AI research news, cool AI projects, and more.

In the event you like our work, you’ll love our newsletter..