Efficient Ingestion, Labeling, and Storage of LiDAR, Radar, and Camera Datasets: Optimizing Storage Formats for Autonomous Vehicle Development

Abstract

The development of autonomous vehicle technologies has placed new standards of managing the multimodal sensor data that includes LiDAR point clouds, radar measurements, and camera images. The contemporary autonomous platforms produce large volumes of non-uniform perception data that need advanced infrastructure to ingest, label, and store in the long term. Combining different modalities of sensing creates significant technical issues throughout the data lifetime, both during the initial capture and archival preservation. Infrastructure Edge preprocessing hardware will need to synchronize heterogeneous sensor streams with less than millisecond accuracy and apply real-time compression to achieve a lower bandwidth in transmission. Automated labeling workflows based on pre-trained perception models can help reduce the manual annotation load by a significant factor, and label quality is likely to be high enough to be used in training. Storage format optimization trades off conflicting demands such as compression performance, random access performance, and compatibility with the distributed processing model. The use of cloud-scale deployment allows managing the petabyte-scale volumes of data and optimizing the costs of infrastructure based on the tiered storage approaches that match the storage performance properties with the access patterns. Benchmark datasets have played a vital role in achieving advances in autonomous vehicle perception, and have allowed the performance to be systematically compared and the outstanding challenges to be reflected. Direct neural network processing of point cloud data has permitted significant progress in performance in detection and segmentation. Further progress will be realised through further growing multi-modal datasets with multi-modes of operation and persistent advances in algorithms that fuse sensor data and build scene understanding.