# Pointcloud map preprocessing

## Overview

Pointcloud map used in Autoware should be downsampled with a voxel size of around 0.1 to 0.2 meters to minimize the computational costs and memory bandwidth load. Also, you may consider using dynamic map loading with partial loading if your map is still too large to handle as a single file (\~2 GB in PCD size). Please check [map\_loader package](<https://github.com/autowarefoundation/autoware_core/tree/main/map/autoware_map_loader>) for more information.

For partial loading, the map must be divided into a grid of PCD files, and it needs a `pointcloud_map_metadata.yaml` file. [`autoware_pointcloud_divider`](<https://github.com/autowarefoundation/autoware_tools/tree/main/map/autoware_pointcloud_divider>) in `autoware_tools` divides a map into cells of 20 m by 20 m by default, downsamples it with a leaf size of 0.2 m by default, and writes the metadata file:

```bash
ros2 launch autoware_pointcloud_divider pointcloud_divider.launch.xml input_pcd_or_dir:=<INPUT_DIR> output_pcd_dir:=<OUTPUT_DIR> prefix:=<PREFIX>
```

Use absolute paths for `<INPUT_DIR>` and `<OUTPUT_DIR>`. The tool deletes all the files in `<OUTPUT_DIR>` before it runs. Thus, do not use your map directory as `<OUTPUT_DIR>`.

The tool writes the divided PCD files to the `<OUTPUT_DIR>/pointcloud_map.pcd/` directory and the metadata to `<OUTPUT_DIR>/pointcloud_map_metadata.yaml`. Copy both into your map directory, next to `lanelet2_map.osm` and `map_projector_info.yaml`. Autoware reads the metadata file from the map directory.

At tutorial\_vehicle implementation we will use the whole map, so we will downsample it with using [CloudCompare](<https://www.cloudcompare.org/main.html>).

## Install CloudCompare

You can install it by snap:

```bash
sudo snap install cloudcompare
```

> [!NOTE]
>
> **Info**
>
> Check out the [official CloudCompare site](<https://www.cloudcompare.org/release/index.html#CloudCompare>) for more installation options.

## Outlier filtering a pointcloud map

We will use SOR (Statistical Outlier Filter) for our mapping process. [Statistical Outlier Removal (SOR)](<https://www.cloudcompare.org/doc/wiki/index.php/SOR_filter>) is a method for detecting and removing anomalous data (points far from their neighbors) points from a data set.

It works by computing the mean and standard deviation of the data points, and then identifying any points that fall outside a certain number of standard deviations from the mean. These points are then flagged as outliers and removed from the pointcloud.

1. Open CloudCompare and drag your point cloud here. Then, you can select your point cloud map by clicking on it in the DB tree panel or on the map.
2. Then click the `SOR` filter button on the top panel.

CloudCompare

- The SOR parameter filters panel will appear. You can change these parameters according to your map specifications. Please refer to [the documentation](<https://www.cloudcompare.org/doc/wiki/index.php/SOR_filter>) for more information about these parameters. We will use the default parameters for our documentation.

SOR Filter parameters

- After the filtering process is finished, you should select pointcloud on the DB Tree panel as well.

Select your cleaned pointcloud

Now, you can save your downsampled pointcloud with <kbd class="key-control">Ctrl</kbd>\+<kbd class="key-s">S</kbd> or you can click save button from `File` menu. Then, this pointcloud can be used by autoware.

| RAW Pointcloud Map | SOR Filtered Pointcloud Map |
| :---: | :---: |
|  |  |

## Downsampling a pointcloud map

There are three [subsampling methods on CloudCompare](<https://www.cloudcompare.org/doc/wiki/index.php/Edit%5CSubsample>), we are using `Space` method for subsampling, but you can use other methods if you want.

1. Please open CloudCompare and drag your point cloud here. Then, you can select your point cloud map by clicking on it in the DB tree panel or on the map.
2. Then you can click `subsample` button on the top panel.

CloudCompare

1. Please select on your subsample method, we will use space for tutorial\_vehicle.
2. Then you can select options. For example, we need to determine minimum space between points. (Please be careful in this section, subsampling is depending on your map size, computer performance, etc.) We will set this value 0.2 for tutorial\_vehicle's map.

Pointcloud subsampling

- Once the subsampling process is finished, you should select pointcloud on the DB Tree panel as well.

Select your downsampled pointcloud

Now, you can save your downsampled pointcloud with <kbd class="key-control">Ctrl</kbd>\+<kbd class="key-s">S</kbd> or you can click save button from `File` bar. Then, this pointcloud can be used by autoware.

| RAW Pointcloud Map | (0.2 Space) Sub-sampled Pointcloud Map |
| :---: | :---: |
|  |  |

## Dynamic object removal

The [dynamic object removal package](<https://github.com/davutcanakbas/dynamic_object_removal>) removes dynamic objects from a point cloud using the detected objects of the lidar CenterPoint package.

> [!WARNING]
>
> This package might be outdated.
>
> Its README uses the old package name `lidar_centerpoint`. In Autoware 2.0.0, the package is `autoware_lidar_centerpoint`.
>
> Please check the repository for the latest updates. If it works well, please make a PR to remove this warning.

> [!NOTE]
>
> **Info**
>
> Please refer to [the package documentation](<https://github.com/davutcanakbas/dynamic_object_removal/blob/main/README.md>) for more information.

> [!NOTE]
>
> **Info**
>
> To learn how to use this package with the LIO-SAM mapping method, follow this documentation:

[Create a LIO-SAM map with dynamic object removal](<https://gist.github.com/StepTurtle/4e74492489073c053883888be57bb1e4>).