Multi-Sensor Data Fusion using DarkNET - CNN

Abstract

Multi-sensor data fusion is ubiquitous; therefore, the associated research is significant. There are several instances in the day-to-day activities where data fusion can be observed. The present generation autonomous driving system requires a thorough understanding followed by a voluminous dataset for training the model. The experimental data of imagery and proximity sensors are significant for the model’s performance. The projection of the camera to LiDAR proves ineffective as the semantic density of the camera is suppressed in the process. The present work attempts to enhance the conventional point-level fusion techniques by allocating prime importance to semantic density. This is facilitated by performance optimization by identifying the hindrances and enhancing the transformation of the view by the Bird’s-eye-View pooling. The object tracking is facilitated through the Extended Kalman Filter(EKF) by fusing the LiDAR data with the camera detections. The detection precision is found to be 0.9546, and the detection recall is 0.9344, while the mAP is evaluated to be 71.2%.