Ultra-High-Frequency Harmony: mmWave Radar and Event Camera Orchestrate Accurate Drone Landing

ACM SenSys 2025 Submissions #48

Anonymous authors

Indoor Experiments

We conduct experiments in an indoor laboratory using the DJI Mini 3 Pro drone, with a motion capture system providing ground truth localization for the drone. mmE-Loc achieves precise, low-latency drone localization by embracing and harmonizing the ultra-high sampling frequencies of mmWave radar and event camera.

Delivery Drone Experiments

We conduct experiments using a custom six-propeller drone, developed by a leading delivery company which is exploring feasibility of instant drone delivery. The drone follows a square spiral trajectory at an altitude of 30m. Results show that, in real-world scenarios, mmE-Loc achieves high localization precision, with maximum absolute location errors under 0.5m and relative location errors under 0.1m. Additionally, mmE-Loc produces smooth trajectories that closely align with RTK, underscoring its potential as a complementary system to RTK.

mmE-Loc enhance Radar with Event Camera for Precise Low Latency Drone Localization


Abstract

For precise, efficient, and safe drone landings, ground platforms should real-time, accurately locate descending drones and guide them to designated spots. While mmWave sensing combined with cameras improves localization accuracy, the lower sampling frequency of traditional frame cameras compared to mmWave radar creates bottlenecks in system throughput. In this work, we replace the traditional frame camera with event camera, a novel sensor that harmonizes in sampling frequency with mmWave radar within the ground platform setup, and introduce mmE-Loc, a high-precision, low-latency ground localization system designed for drone landings. To fully leverage the temporal consistency and spatial complementarity between these modalities, we propose two innovative modules, consistency-instructed collaborative tracking and graph-informed adaptive joint optimization, for accurate drone measurement extraction and efficient sensor fusion. Extensive real-world experiments in landing scenarios from a leading drone delivery company demonstrate that mmE-Loc outperforms state-of-the-art methods in both localization accuracy and latency.

Event Camera Preliminary

Event cameras are bio-inspired sensors that report pixel-wise intensity changes with ms-level resolution, capturing high-speed motions without blurring, ideal for fast-tracking tasks. Event cameras offer ms-level sampling latency, which harmonizes exceptionally with the high sampling frequencies of mmWave radar. Their 2D imaging capability also complements radars' limited spatial resolution, similar to how traditional frame cameras operate. Such temporal consistency and spatial complementarity across both modalities inspire us to upgrade frame cameras with event cameras to pair with radar for accurate and fast drone localization.

Image 1

System Overview

The mmE-Loc comprises two key modules:

(i) The CCT (Consistency-instructed Collaborative Tracking) for noise filtering, drone detection, preliminary localization of the drone. This module utilizes temporal consistency of both modalities and drone's periodic micro-motion to eliminate noise and error detections, and preliminary localize the drone.

(ii) The GAJO (Graph-informed Adaptive Joint Optimization) for fine localization and trajectory optimization of the drone. This module employs the spatial complementarity from both modalities to unleash the potential of event camera and mmWave radar in landing drone localization.

Image 1

Implementation and Experiments Setup

We implement mmE-Loc with a handheld prototype equipped with multiple sensors including (i) A Prophesee EVK4 HD evaluation kit, featuring the IMX636ES event-based vision sensor for HD event data (1280 $\times$ 720 pixels) with the Soyo SFA0820-5M lens. (ii) A Texas Instruments (TI) IWR1843 board for transmitting and receiving mmWave signals within the frequency range of 76 GHz to 81 GHz. The TI IWR1843 integrates three transmitting antennas labeled TX1 to TX3, and four receiving antennas labeled RX1 to RX4. These antennas are arranged in two linear configurations on the horizontal plane. (iii) An Intel D435i Depth camera for RGB image capture used in the baseline method. All the sensors are synchronized through Robot Operating System (ROS). The mmE-Loc operates on a PC equipped with Ubuntu 20.04, an Intel i7-12900K CPU, and 32GB RAM.
Image 1

Quantitative results

Overall Performances

Image 1

Robustness Evaluation

Image 1

Ablation Study

Image 1

System Overload

Image 1

Airport Evaluation

Real-world delivery drone airpot evaluation result

Image 1

Experiments Video