[3d object detection] waymo_dataset visualize

[이승현]

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96

CLASS_NAMES: ['Vehicle', 'Pedestrian', 'Cyclist']   DATA_CONFIG:     _BASE_CONFIG_: cfgs/dataset_configs/waymo_dataset.yaml   MODEL:     NAME: CenterPoint       VFE:         NAME: MeanVFE       BACKBONE_3D:         NAME: VoxelResBackBone8x       MAP_TO_BEV:         NAME: HeightCompression         NUM_BEV_FEATURES: 256       BACKBONE_2D:         NAME: BaseBEVBackbone           LAYER_NUMS: [5, 5]         LAYER_STRIDES: [1, 2]         NUM_FILTERS: [128, 256]         UPSAMPLE_STRIDES: [1, 2]         NUM_UPSAMPLE_FILTERS: [256, 256]       DENSE_HEAD:         NAME: CenterHead         CLASS_AGNOSTIC: False           CLASS_NAMES_EACH_HEAD: [             ['Vehicle', 'Pedestrian', 'Cyclist']         ]           SHARED_CONV_CHANNEL: 64         USE_BIAS_BEFORE_NORM: True         NUM_HM_CONV: 2         SEPARATE_HEAD_CFG:             HEAD_ORDER: ['center', 'center_z', 'dim', 'rot']             HEAD_DICT: {                 'center': {'out_channels': 2, 'num_conv': 2},                 'center_z': {'out_channels': 1, 'num_conv': 2},                 'dim': {'out_channels': 3, 'num_conv': 2},                 'rot': {'out_channels': 2, 'num_conv': 2},             }           TARGET_ASSIGNER_CONFIG:             FEATURE_MAP_STRIDE: 8             NUM_MAX_OBJS: 500             GAUSSIAN_OVERLAP: 0.1             MIN_RADIUS: 2           LOSS_CONFIG:             LOSS_WEIGHTS: {                 'cls_weight': 1.0,                 'loc_weight': 2.0,                 'code_weights': [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]             }           POST_PROCESSING:             SCORE_THRESH: 0.25 # 0.1             POST_CENTER_LIMIT_RANGE: [-75.2, -75.2, -2, 75.2, 75.2, 4]             MAX_OBJ_PER_SAMPLE: 500             NMS_CONFIG:                 NMS_TYPE: nms_gpu                 NMS_THRESH: 0.6 # 0.7                 NMS_PRE_MAXSIZE: 4096                 NMS_POST_MAXSIZE: 500       POST_PROCESSING:         RECALL_THRESH_LIST: [0.3, 0.5, 0.7]           EVAL‎_METRIC: waymo     OPTIMIZATION:     BATCH_SIZE_PER_GPU: 4     NUM_EPOCHS: 80       OPTIMIZER: adam_onecycle     LR: 0.003     WEIGHT_DECAY: 0.01     MOMENTUM: 0.9       MOMS: [0.95, 0.85]     PCT_START: 0.4     DIV_FACTOR: 10     DECAY_STEP_LIST: [35, 45]     LR_DECAY: 0.1     LR_CLIP: 0.0000001       LR_WARMUP: False     WARMUP_EPOCH: 1       GRAD_NORM_CLIP: 10 Colored by Color Scripter

cs

nms 조정 0.7--> 0.6 score_thresh 0.1 -->0.25 한 결과 많은 객체를 찾지는 못하지만 겹치는 바운딩 박스가 줄고 정확도가 올라감

논문의 validation 성능

시각화 코드 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80

import open3d as o3d import numpy as np import pickle import os   box_colormap = {     1: [0, 1, 0],    # Vehicle      2: [0, 1, 1],    # Pedestrian      3: [1, 1, 0],    # Cyclist  }   def draw_scenes(points, gt_boxes=None, ref_boxes=None, ref_labels=None):     vis = o3d.visualization.Visualizer()     vis.create_window(width=1920, height=1080)     opt = vis.get_render_option()     opt.background_color = np.zeros(3)  #배경     opt.point_size = 1.0       pcd = o3d.geometry.PointCloud()     pcd.points = o3d.utility.Vector3dVector(points[:, :3])     pcd.colors = o3d.utility.Vector3dVector(np.ones((points.shape[0], 3)))     vis.add_geometry(pcd)       # GT Box      if gt_boxes is not None and len(gt_boxes) > 0:         vis = draw_box(vis, gt_boxes, color=(0, 0, 1))       # Pred Box     if ref_boxes is not None and len(ref_boxes) > 0:         vis = draw_box(vis, ref_boxes, ref_labels=ref_labels)       vis.run()     vis.destroy_window()   def draw_box(vis, boxes, color=(0, 1, 0), ref_labels=None):     for i in range(boxes.shape[0]):         line_set, _ = create_box_line_set(boxes[i])         if ref_labels is not None:             color = box_colormap.get(ref_labels[i], (1, 1, 1))         line_set.paint_uniform_color(color)         vis.add_geometry(line_set)     return vis   def create_box_line_set(box):     center = box[:3]     size = box[3:6]     heading = box[6]     axis_angles = [0, 0, heading]     R = o3d.geometry.get_rotation_matrix_from_axis_angle(axis_angles)     obb = o3d.geometry.OrientedBoundingBox(center, R, size)     line_set = o3d.geometry.LineSet.create_from_oriented_bounding_box(obb)     return line_set, obb   def process_single_frame(pointcloud_path, result_pkl_path, info_pkl_path, frame_index):     pointcloud = np.load(pointcloud_path)       with open(result_pkl_path, 'rb') as f:         preds = pickle.load(f)       with open(info_pkl_path, 'rb') as f:         infos = pickle.load(f)       gt_boxes = infos[frame_index]['annos']['gt_boxes_lidar']       pred_frame = preds[frame_index]     ref_boxes = pred_frame['boxes_lidar']     ref_labels = pred_frame['name']       label_map = {"Vehicle": 1, "Pedestrian": 2, "Cyclist": 3}     ref_labels = [label_map.get(label, 0) for label in ref_labels]       draw_scenes(pointcloud, gt_boxes=gt_boxes, ref_boxes=ref_boxes, ref_labels=ref_labels)   if __name__ == "__main__":     result_pkl = r"C:\Users\airlab\Desktop\ex_3result.pkl"     info_pkl = r"C:\Users\airlab\Desktop\waymo_processed_data_v0_5_0_infos_val.pkl"     pointcloud_npy = r"D:\waymo\waymo_processed_data_v0_5_0\segment-10203656353524179475_7625_000_7645_000_with_camera_labels\0000.npy"     frame_idx = 0     process_single_frame(pointcloud_npy, result_pkl, info_pkl, frame_idx)   Colored by Color Scripter

cs

1 2 3 4 5 6 7 8 9

2025-04-02 16:02:55,726   INFO  Generate label finished(sec_per_example: 0.0263 second). 2025-04-02 16:02:55,726   INFO  recall_roi_0.3: 0.000000 2025-04-02 16:02:55,726   INFO  recall_rcnn_0.3: 0.812775 2025-04-02 16:02:55,726   INFO  recall_roi_0.5: 0.000000 2025-04-02 16:02:55,726   INFO  recall_rcnn_0.5: 0.745850 2025-04-02 16:02:55,726   INFO  recall_roi_0.7: 0.000000 2025-04-02 16:02:55,726   INFO  recall_rcnn_0.7: 0.485659 2025-04-02 16:02:55,738   INFO  Average predicted number of objects(38597 samples): 55.901   Colored by Color Scripter

cs

validation 시각화

1 2 3 4 5 6 7 8

2025-04-02 16:29:26,586   INFO  Generate label finished(sec_per_example: 0.0295 second). 2025-04-02 16:29:26,586   INFO  recall_roi_0.3: 0.000000 2025-04-02 16:29:26,586   INFO  recall_rcnn_0.3: 0.875266 2025-04-02 16:29:26,586   INFO  recall_roi_0.5: 0.000000 2025-04-02 16:29:26,586   INFO  recall_rcnn_0.5: 0.841705 2025-04-02 16:29:26,586   INFO  recall_roi_0.7: 0.000000 2025-04-02 16:29:26,586   INFO  recall_rcnn_0.7: 0.649405 2025-04-02 16:29:26,597   INFO  Average predicted number of objects(30705 samples): 58.204

cs

다음주 목표

1. 3D모델 SOTA( state-of-the-art) 찾기

2. 훈련 데이터셋에 대한 시각화 (검증과 비교하여 잘 찾는지 비교)

[Sungryul Lee]

영상에 시각화 추가할것, 정면 영상에만 그려볼것