[딥러닝] 24. 자율주행 시뮬레이터 Carla

<p>- 자율주행 실험을 실제 도로에서 하기 힘드므로 시뮬레이터를 이용하여 실제와 유사한 환경에서 자율주행 알고리즘을 테스트 가능</p><p><span style="color: #333333; background-color: #ffffff;" data-ke-size="size16">- </span> 가상 카메라, 가상 라이다 등 모든 센서와 액추에이터, 자동차모델, 도로맵 등 라이브러리제공 -> 실제 고가의 센서 및 로봇이 필요없이 가상공간에서 맘대로 실험해볼수 있음</p><p><span style="color: #333333; background-color: #ffffff;" data-ke-size="size16">- </span> ROS bridge기능 제공하여 ROS에서도 바로 제어가능 -> 0.10.0버전부터 ROS 토픽 생성기능 내장되어 ROS bridge 필요없음</p><p><span style="color: #333333; background-color: #ffffff;" data-ke-size="size16">- </span> 실제와 거의 같은 환경에서 다양한 알고리즘 테스트 가능</p><p> </p><p>1. Carla</p><p><a href="https://carla.org/" target="_blank" class="ke-link">https://carla.org/</a></p><div class="figure-open" contenteditable="false" data-ke-type="opengraph" data-ke-align="alignCenter" data-og-type="website" data-og-title="CARLA" data-og-description="Open-source simulator for autonomous driving research." data-og-host="carla.org" data-og-source-url="https://carla.org/" data-og-url="http://carla.org//" data-og-image="https://scrap.kakaocdn.net/dn/RnLxo/hyW2TBq5KQ/k6mB2zMTd1floNCEM5dnU1/img.jpg?width=512&height=384&face=0_0_512_384,https://scrap.kakaocdn.net/dn/SPm4l/hyW2SJidk9/ysn360gLDEl8LIMYZHUkik/img.jpg?width=512&height=384&face=0_0_512_384,https://scrap.kakaocdn.net/dn/cwqheE/hyWZkUZteL/nqhmDw4gDfI0JxXalwPhu1/img.png?width=1743&height=449&face=0_0_1743_449"><a href="https://carla.org/" target="_blank" data-source-url="https://carla.org/"><div class="og-image"><img class="thumb_img" src="https://scrap.kakaocdn.net/dn/RnLxo/hyW2TBq5KQ/k6mB2zMTd1floNCEM5dnU1/img.jpg?width=512&height=384&face=0_0_512_384,https://scrap.kakaocdn.net/dn/SPm4l/hyW2SJidk9/ysn360gLDEl8LIMYZHUkik/img.jpg?width=512&height=384&face=0_0_512_384,https://scrap.kakaocdn.net/dn/cwqheE/hyWZkUZteL/nqhmDw4gDfI0JxXalwPhu1/img.png?width=1743&height=449&face=0_0_1743_449" alt="" xxxxonerror="this.src="//img1.kakaocdn.net/thumb/C200x200/?fname=https%3A%2F%2Ft1.daumcdn.net%2Fcafe_image%2Fcafe_meta_image_190529.png""></div><div class="og-text"><p class="og-title">CARLA</p><p class="og-desc">Open-source simulator for autonomous driving research.</p><p class="og-host">carla.org</p></div></a></div><p> </p><p>2. Three Methods of Vehicle Lateral Control: Pure Pursuit, Stanley and MPC Theory</p><p><a href="https://dingyan89.medium.com/three-methods-of-vehicle-lateral-control-pure-pursuit-stanley-and-mpc-db8cc1d32081" target="_blank" class="ke-link">https://dingyan89.medium.com/three-methods-of-vehicle-lateral-control-pure-pursuit-stanley-and-mpc-db8cc1d32081</a></p><p><a href="https://www.shuffleai.blog/blog/Three_Methods_of_Vehicle_Lateral_Control.html" target="_blank" class="ke-link">https://www.shuffleai.blog/blog/Three_Methods_of_Vehicle_Lateral_Control.html</a></p><p> </p><p>github</p><p><a href="https://github.com/casper-auto/carla_trajectory_controller" target="_blank" class="ke-link">https://github.com/casper-auto/carla_trajectory_controller</a></p><p> </p><p>Youtube</p><p><a href="https://www.youtube.com/watch?v=O5npCPhOF-s" target="_blank" class="ke-link">https://www.youtube.com/watch?v=O5npCPhOF-s</a></p><p> </p><p>3. carla 0.10.0 설치방법</p><p>carla 0.10.0 버전 설치 -> UE5 사용한 최신 버전 -> map의 화질 대폭개선됨</p><p><a href="https://carla-ue5.readthedocs.io/en/latest/start_quickstart/" target="_blank" class="ke-link">https://carla-ue5.readthedocs.io/en/latest/start_quickstart/</a></p><p> </p><p>4. carla example code실행방법</p><p>carla server 실행 -> python generate_traffic.py(차량, 보행자 생성해줌) -> 원하는 예제코드실행</p><p>rtx 3070에서 약 7~9fps 처리속도 나옴</p><p>rtx 4070 ti super에서 30fps 이상 나옴</p><p> </p><p>5. camera 영상을 파일로 저장하는 예제</p><p>예제코드에서 카메라 프레임을 jpg로 저장(my_automatic_control.py) -> 여러개의 jpg를 mp4로 변환저장(my_jpgs2mp4.py) -> 영상또는 동영상을 실습에 사용</p><div class="figure-file" data-ke-type="file" data-file-src="https://t1.daumcdn.net/cafeattach/1RgNt/07d87fdb258e6a88cf7c7a37eba27eb639011c63?download" data-file-name="자율주행로봇체험_230712.pdf" data-file-size="1374324" data-mimetype="application/pdf" data-ke-align="alignCenter"><a href="javascript:checkVirus('grpid%3D1RgNt%26fldid%3DW2or%26dataid%3D177%26fileid%3D1%26regdt%3D20240908211427&url=https%3A%2F%2Ft1.daumcdn.net%2Fcafeattach%2F1RgNt%2F07d87fdb258e6a88cf7c7a37eba27eb639011c63')"><div class="image"></div><div class="desc"><div class="filename"><span class="name">자율주행로봇체험_230712</span><span class="type">.pdf</span></div><div class="size">1.31MB</div></div></a></div><div class="figure-file" data-ke-type="file" data-file-src="https://t1.daumcdn.net/cafeattach/1RgNt/296620339c878e97e667929937c3207c6be42658?download" data-file-name="my_automatic_control.py" data-file-size="39857" data-mimetype="" data-ke-align="alignCenter"><a href="javascript:checkVirus('grpid%3D1RgNt%26fldid%3DW2or%26dataid%3D177%26fileid%3D3%26regdt%3D20240908211427&url=https%3A%2F%2Ft1.daumcdn.net%2Fcafeattach%2F1RgNt%2F296620339c878e97e667929937c3207c6be42658')"><div class="image"></div><div class="desc"><div class="filename"><span class="name">my_automatic_control</span><span class="type">.py</span></div><div class="size">38.92KB</div></div></a></div><div class="figure-file" data-ke-type="file" data-file-src="https://t1.daumcdn.net/cafeattach/1RgNt/5e33fa81ea376934f97a86fce4a6046397e933aa?download" data-file-name="my_jpgs2mp4.py" data-file-size="393" data-mimetype="" data-ke-align="alignCenter"><a href="javascript:checkVirus('grpid%3D1RgNt%26fldid%3DW2or%26dataid%3D177%26fileid%3D2%26regdt%3D20240908211427&url=https%3A%2F%2Ft1.daumcdn.net%2Fcafeattach%2F1RgNt%2F5e33fa81ea376934f97a86fce4a6046397e933aa')"><div class="image"></div><div class="desc"><div class="filename"><span class="name">my_jpgs2mp4</span><span class="type">.py</span></div><div class="size">393B</div></div></a></div><p>6. lidar 데이터를 파일로 저장하는 예제</p><p>3d point cloud 데이터 처리를 위해 open3d 사용</p><p><a href="https://www.open3d.org/" target="_blank" class="ke-link">https://www.open3d.org/</a></p><div class="figure-file" data-ke-type="file" data-file-src="https://t1.daumcdn.net/cafeattach/1RgNt/b73d2005e6d0ecea3113b6acaeba56f954d6aa23?download" data-file-name="my_visualize_multiple_sensors.py" data-file-size="17604" data-mimetype="" data-ke-align="alignCenter"><a href="javascript:checkVirus('grpid%3D1RgNt%26fldid%3DW2or%26dataid%3D177%26fileid%3D4%26regdt%3D20240908211427&url=https%3A%2F%2Ft1.daumcdn.net%2Fcafeattach%2F1RgNt%2Fb73d2005e6d0ecea3113b6acaeba56f954d6aa23')"><div class="image"></div><div class="desc"><div class="filename"><span class="name">my_visualize_multiple_sensors</span><span class="type">.py</span></div><div class="size">17.19KB</div></div></a></div><p> </p><div class="figure-file" data-ke-type="file" data-file-src="https://t1.daumcdn.net/cafeattach/1RgNt/2ed42fe53ae4b404957e5fe9959a84ab7d3e712d?download" data-file-name="my_pcshow.py" data-file-size="883" data-mimetype="" data-ke-align="alignCenter"><a href="javascript:checkVirus('grpid%3D1RgNt%26fldid%3DW2or%26dataid%3D177%26fileid%3D5%26regdt%3D20240908211427&url=https%3A%2F%2Ft1.daumcdn.net%2Fcafeattach%2F1RgNt%2F2ed42fe53ae4b404957e5fe9959a84ab7d3e712d')"><div class="image"></div><div class="desc"><div class="filename"><span class="name">my_pcshow</span><span class="type">.py</span></div><div class="size">883B</div></div></a></div><p>7. <span data-ke-size="size16"><span style="color: #1f2328; background-color: #ffffff;">Data collection tools for CARLA Simulator.</span> </span></p><p><span data-ke-size="size16"><span style="color: #1f2328; background-color: #ffffff;">carla 시뮬레이터에서 다양한 센서로부터 훈련용 데이터를 수집하는 소스코드들</span></span></p><p><span data-ke-size="size16"><span style="color: #1f2328; background-color: #ffffff;">예를 들어, carla lidar 센서에서 측정된 데이터를 KITTI dataset 포맷으로 저장해주는 예제 등</span></span></p><p><a href="https://github.com/KevinLADLee/carla_dataset_tools" target="_blank" class="ke-link">https://github.com/KevinLADLee/carla_dataset_tools</a></p><p><a href="https://github.com/SIAT-INVS/CarlaFLCAV" target="_blank" class="ke-link">https://github.com/SIAT-INVS/CarlaFLCAV</a></p><p><a href="https://wandb.ai/makaveli/3D-Perception-Results/reports/3D-Perception-in-Carla-Simulator--VmlldzoyNTYwMTcy" target="_blank" class="ke-link">https://wandb.ai/makaveli/3D-Perception-Results/reports/3D-Perception-in-Carla-Simulator--VmlldzoyNTYwMTcy</a></p><p> </p><p>8. Carla 라이브러리</p><p><a href="https://carla-ue5.readthedocs.io/en/latest/python_api/#carla.BoundingBox" target="_blank" class="ke-link">https://carla-ue5.readthedocs.io/en/latest/python_api/#carla.BoundingBox</a></p><p>-> Carla 환경의 모든 객체의 3차원 바운딩 박스정보를 알수있음(로컬좌표계 또는 월드좌표계)</p><p> </p><p><a href="https://carla-ue5.readthedocs.io/en/latest/ref_sensors/#semantic-lidar-sensor" target="_blank" class="ke-link">https://carla-ue5.readthedocs.io/en/latest/ref_sensors/#semantic-lidar-sensor</a> </p><p>-> 라이다 시멘틱 라이다 정보를 제공, 여기서 라이다센서의 위치와 포인트클라우드 데이터를 얻을수 있음</p><p>-> 위의 3차원 바운딩박스정보를 이용하여 월드좌표계와 라이다좌표계의 사이의 좌표변환을 이용하면 라이다좌표계에서 객체의 3d 바운딩 박스정보를 얻을수 있음</p><p>-> 포인트 클라우드 데이터의 3d bbox label(ground truth)을 얻을수 있음</p><p> </p><p>비슷한 방식으로 carla 라이브러리를 이용하면 신호등의 바운딩박스정보와 신호등 상태정보를 얻을수 있을것 같음</p>