[3d object detection] LION 모델 Waymo 데이터셋 추론 및 시각화(진행중)

[이승현]

사용환경

• OS: Ubuntu 20.04 (WSL2)
• GPU: NVIDIA RTX 3070
• python 3.8
• CUDA 11.8

성능표

LION NuScenes 사전학습 모델 사용

클래스 맵핑 적용:

• 원래 NuScenes: 클래스 10개

• Waymo: 클래스 3개 (Vehicle, Pedestrian, Cyclist)

• 추론 시 내부적으로 클래스 매핑 처리함

1 2 3 4 5 6 7 8 9 10

2025-05-05 01:10:49,353   INFO  *************** EPOCH 36 EVAL‎UATION ***************** 2025-05-05 01:53:30,725   INFO  *************** Performance of EPOCH 36 ***************** 2025-05-05 01:53:30,726   INFO  Generate label finished(sec_per_example: 0.4966 second). 2025-05-05 01:53:30,726   INFO  recall_roi_0.3: 0.000000 2025-05-05 01:53:30,726   INFO  recall_rcnn_0.3: 0.000727 2025-05-05 01:53:30,726   INFO  recall_roi_0.5: 0.000000 2025-05-05 01:53:30,726   INFO  recall_rcnn_0.5: 0.000004 2025-05-05 01:53:30,726   INFO  recall_roi_0.7: 0.000000 2025-05-05 01:53:30,726   INFO  recall_rcnn_0.7: 0.000000 2025-05-05 01:53:30,728   INFO  Average predicted number of objects(5158 samples): 500.000

cs

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170

CLASS_NAMES: ['car', 'truck', 'bus', 'trailer', 'construction_vehicle',               'pedestrian', 'motorcycle', 'bicycle', 'barrier', 'traffic_cone']  # 원래 NuScenes 클래스 목록   DATA_CONFIG:   _BASE_CONFIG_: cfgs/dataset_configs/waymo_dataset.yaml     DATA_PATH: '/home/linux/OpenPCDet/data/waymo'   POINT_CLOUD_RANGE: [-75.52, -75.52, -2, 75.52, 75.52, 4.0]     POINT_FEATURE_ENCODING:     encoding_type: absolute_coordinates_encoding     used_feature_list: ['x', 'y', 'z', 'intensity', 'timestamp']       src_feature_list: ['x', 'y', 'z', 'intensity', 'timestamp']         DATA_AUGMENTOR:     DISABLE_AUG_LIST: ['placeholder']     AUG_CONFIG_LIST: []     DATA_PROCESSOR:     - NAME: mask_points_and_boxes_outside_range       REMOVE_OUTSIDE_BOXES: True       - NAME: shuffle_points       SHUFFLE_ENABLED: {         'train': True,         'test': True       }       - NAME: transform_points_to_voxels_placeholder       VOXEL_SIZE: [0.32, 0.32, 0.1875]   MODEL:   NAME: TransFusion     VFE:     NAME: DynamicVoxelVFE     WITH_DISTANCE: False     USE_ABSLOTE_XYZ: True     USE_NORM: True     NUM_FILTERS: [128, 128]     NUM_POINT_FEATURES: 5   # ✅ 수정 (4 → 5)     BACKBONE_3D:     NAME: LION3DBackboneOneStride     FEATURE_DIM: 128     LAYER_DIM: [128, 128, 128, 128]     NUM_LAYERS: 4     DEPTHS: [2, 2, 2, 2]     LAYER_DOWN_SCALES: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]     WINDOW_SHAPE: [[13, 13, 32], [13, 13, 16], [13, 13, 8], [13, 13, 4]]     GROUP_SIZE: [4096, 2048, 1024, 512]     DIRECTION: ['x', 'y']     DIFF_SCALE: 0.2     DIFFUSION: True     SHIFT: True     OPERATOR:       NAME: 'Mamba'       CFG:         d_state: 16         d_conv: 4         expand: 2         drop_path: 0.2     MAP_TO_BEV:     NAME: HeightCompression     NUM_BEV_FEATURES: 256     BACKBONE_2D:     NAME: BaseBEVResBackbone     LAYER_NUMS: [1, 2, 2]     LAYER_STRIDES: [1, 2, 2]     NUM_FILTERS: [128, 128, 256]     UPSAMPLE_STRIDES: [0.5, 1, 2]     NUM_UPSAMPLE_FILTERS: [128, 128, 128]     DENSE_HEAD:     CLASS_AGNOSTIC: False     NAME: TransFusionHead       USE_BIAS_BEFORE_NORM: False       NUM_PROPOSALS: 200     HIDDEN_CHANNEL: 128     NUM_CLASSES: 10     NUM_HEADS: 8     NMS_KERNEL_SIZE: 3     FFN_CHANNEL: 256     DROPOUT: 0     BN_MOMENTUM: 0.1     ACTIVATION: relu       NUM_HM_CONV: 2     SEPARATE_HEAD_CFG:       HEAD_ORDER: ['center', 'height', 'dim', 'rot']       HEAD_DICT:         center: {out_channels: 2, num_conv: 2}         height: {out_channels: 1, num_conv: 2}         dim: {out_channels: 3, num_conv: 2}         rot: {out_channels: 2, num_conv: 2}         iou: {out_channels: 1, num_conv: 2}       TARGET_ASSIGNER_CONFIG:       # HUNGARIAN_ASSIGNER:       #   cls_cost:       #     weight: 1.0       #   reg_cost:       #     weight: 1.0       #   iou_cost:       #     weight: 1.0       #   iou_calculator:       #     type: iou3d_calculator       #   max_pos: 15       FEATURE_MAP_STRIDE: 2       DATASET: waymo       GAUSSIAN_OVERLAP: 0.1       MIN_RADIUS: 2       LOSS_CONFIG:       LOSS_WEIGHTS:         cls_weight: 1.0         bbox_weight: 0.25         hm_weight: 1.0         loss_iou_rescore_weight: 0.5         code_weights: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]       LOSS_CLS:         use_sigmoid: True         gamma: 2.0         alpha: 0.25       POST_PROCESSING:       SCORE_THRESH: 0.1       POST_CENTER_RANGE: [-80, -80, -10.0, 80, 80, 10.0]       USE_IOU_TO_RECTIFY_SCORE: True       IOU_RECTIFIER: [0.68, 0.71, 0.65]       NMS_CONFIG:         NMS_TYPE: nms_gpu         NMS_THRESH: 0.2         NMS_PRE_MAXSIZE: 1000         NMS_POST_MAXSIZE: 100         SCORE_THRES: 0.0   POST_PROCESSING:   RECALL_THRESH_LIST: [0.3, 0.5, 0.7]   SCORE_THRESH: 0.1   OUTPUT_RAW_SCORE: False   EVAL‎_METRIC: waymo   OPTIMIZATION:   BATCH_SIZE_PER_GPU: 2   NUM_EPOCHS: 36   OPTIMIZER: adam_onecycle   LR: 0.003   WEIGHT_DECAY: 0.05   MOMENTUM: 0.9   MOMS: [0.95, 0.85]   PCT_START: 0.4   DIV_FACTOR: 10   DECAY_STEP_LIST: [35, 45]   LR_DECAY: 0.1   LR_CLIP: 0.0000001   LR_WARMUP: False   WARMUP_EPOCH: 1   GRAD_NORM_CLIP: 35   LOSS_SCALE_FP16: 4.0   HOOK:   DisableAugmentationHook:     DISABLE_AUG_LIST: ['gt_sampling']     NUM_LAST_EPOCHS: 4   Colored by Color Scripter

cs

결과 :

• 평균 recall_rcnn_0.3 값: 약 0.0007로 매우 낮음

• Waymo 평가기 실행됨 (detection_metrics_ops.cc)---> 4시간 넘개 평가하였지만 실패
  → 약 257만개의 예측 박스, GT 약 24만 개 비교

계획:

• result.pkl 시각화를 위한 Open3D 기반 스크립트 작성 

• NuScenes 사전학습 모델을 Waymo 데이터셋에 finetune 하여 성능 개선 시도

• 클래스 맵핑하여 waymo에 맞게 재구성