UniqAI/gaussian-splatting
1
0
Fork 0
mirror of https://github.com/graphdeco-inria/gaussian-splatting synced 2024-11-22 00:08:02 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Clearer conversion explanation

Browse Source
This commit is contained in:
bkerbl 2023-07-10 17:58:17 +02:00
parent b9fa8d1508
commit c1b61aad15
2 changed files with 81 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

72
README.md
View File

@ -320,16 +320,16 @@ The network viewer allows you to connect to a running training process on the sa
<details> <details>
<summary><span style="font-weight: bold;">Primary Command Line Arguments for Network Viewer</span></summary> <summary><span style="font-weight: bold;">Primary Command Line Arguments for Network Viewer</span></summary>
### --path / -s #### --path / -s
Argument to override model's path to source dataset. Argument to override model's path to source dataset.
### --ip #### --ip
IP to use for connection to a running training script. IP to use for connection to a running training script.
### --port #### --port
Port to use for connection to a running training script. Port to use for connection to a running training script.
### --rendering-size #### --rendering-size
Takes two space separated numbers to define the resolution at which network rendering occurs, ```1200``` width by default. Takes two space separated numbers to define the resolution at which network rendering occurs, ```1200``` width by default.
Note that to enforce an aspect that differs from the input images, you need ```--force-aspect-ratio``` too. Note that to enforce an aspect that differs from the input images, you need ```--force-aspect-ratio``` too.
### --load_images #### --load_images
Flag to load source dataset images to be displayed in the top view for each camera. Flag to load source dataset images to be displayed in the top view for each camera.
</details> </details>
<br> <br>
@ -359,28 +359,74 @@ SIBR has many other functionalities, please see the [documentation](https://sibr
<details> <details>
<summary><span style="font-weight: bold;">Primary Command Line Arguments for Real-Time Viewer</span></summary> <summary><span style="font-weight: bold;">Primary Command Line Arguments for Real-Time Viewer</span></summary>
### --model-path / -m #### --model-path / -m
Path to trained model. Path to trained model.
### --iteration #### --iteration
Specifies which of state to load if multiple are available. Defaults to latest available iteration. Specifies which of state to load if multiple are available. Defaults to latest available iteration.
### --path / -s #### --path / -s
Argument to override model's path to source dataset. Argument to override model's path to source dataset.
### --rendering-size #### --rendering-size
Takes two space separated numbers to define the resolution at which real-time rendering occurs, ```1200``` width by default. Note that to enforce an aspect that differs from the input images, you need ```--force-aspect-ratio``` too. Takes two space separated numbers to define the resolution at which real-time rendering occurs, ```1200``` width by default. Note that to enforce an aspect that differs from the input images, you need ```--force-aspect-ratio``` too.
### --load_images #### --load_images
Flag to load source dataset images to be displayed in the top view for each camera. Flag to load source dataset images to be displayed in the top view for each camera.
### --device #### --device
Index of CUDA device to use for rasterization if multiple are available, ```0``` by default. Index of CUDA device to use for rasterization if multiple are available, ```0``` by default.
</details> </details>
<br> <br>
## Preprocessing your own Scenes ## Preprocessing your own Scenes
We provide a converter script ```convert.py```, which uses COLMAP to extract SfM information. Optionally, you can use ImageMagick to resize the undistorted images. This rescaling is similar to MipNeRF360, i.e., it creates images with 1/2, 1/4 and 1/8 the original resolution in corresponding folders. To use them, please first install a recent version of COLMAP (ideally CUDA-powered) and ImageMagick. Put the images you want to use in a directory ```<location>/input```. If you have COLMAP and ImageMagick on your system path, you can simply run Our rasterization requires a SIMPLE_PINHOLE or PINHOLE camera model for COLMAP data. We provide a converter script ```convert.py```, to extract undistorted images and SfM information. Optionally, you can use ImageMagick to resize the undistorted images. This rescaling is similar to MipNeRF360, i.e., it creates images with 1/2, 1/4 and 1/8 the original resolution in corresponding folders. To use them, please first install a recent version of COLMAP (ideally CUDA-powered) and ImageMagick. Put the images you want to use in a directory ```<location>/input```.
```
<location>
|---input
| |---<image 0>
| |---<image 1>
| |---...
```
If you have COLMAP and ImageMagick on your system path, you can simply run
```shell ```shell
python convert.py -s <location> [--resize] #If not resizing, ImageMagick is not needed python convert.py -s <location> [--resize] #If not resizing, ImageMagick is not needed
``` ```
Alternatively, you can use the optional parameters ```--colmap_executable``` and ```--magick_executable``` to point to the respective paths. Please note that on Windows, the executable should point to the COLMAP ```.bat``` file that takes care of setting the execution environment. Once done, ```<location>``` will contain the expected COLMAP data set structure with undistorted, differently sized input images, in addition to your original images and temporary data in the directory ```distorted```. Alternatively, you can use the optional parameters ```--colmap_executable``` and ```--magick_executable``` to point to the respective paths. Please note that on Windows, the executable should point to the COLMAP ```.bat``` file that takes care of setting the execution environment. Once done, ```<location>``` will contain the expected COLMAP data set structure with undistorted, resized input images, in addition to your original images and some temporary (distorted) data in the directory ```distorted```.
If you have your own COLMAP dataset without undistortion (e.g., using ```OPENCV``` camera), you can try to just run the last part of the script: Put the images in ```input``` and the COLMAP info in a subdirectory ```distorted```:
```
<location>
|---input
| |---<image 0>
| |---<image 1>
| |---...
|---distorted
| |---sparse
| | |---0
| |---database.db
```
Then run
```shell
python convert.py -s <location> --skip_matching [--resize] #If not resizing, ImageMagick is not needed
```
<details>
<summary><span style="font-weight: bold;">Command Line Arguments for convert.py</span></summary>
#### --no_gpu
Flag to avoid using GPU in COLMAP.
#### --skip_matching
Flag to indicate that COLMAP info is available for images.
#### --source_path / -s
Location of the inputs.
#### --camera
Which camera model to use for the early matching steps, ```OPENCV``` by default.
#### --resize
Flag for creating resized versions of input images.
#### --colmap_executable
Path to the COLMAP executable (```.bat``` on Windows).
#### --magick_executable
Path to the ImageMagick executable.
</details>
<br>
## FAQ ## FAQ
- *Where do I get data sets, e.g., those referenced in ```full_eval.py```?* The MipNeRF360 data set is provided by the authors of the original paper on the project site. Note that two of the data sets cannot be openly shared and require you to consult the authors directly. For Tanks&Temples and Deep Blending, please use the download links provided at the top of the page. - *Where do I get data sets, e.g., those referenced in ```full_eval.py```?* The MipNeRF360 data set is provided by the authors of the original paper on the project site. Note that two of the data sets cannot be openly shared and require you to consult the authors directly. For Tanks&Temples and Deep Blending, please use the download links provided at the top of the page.

42
convert.py
View File

@ -16,6 +16,7 @@ import shutil
# This Python script is based on the shell converter script provided in the MipNerF 360 repository. # This Python script is based on the shell converter script provided in the MipNerF 360 repository.
parser = ArgumentParser("Colmap converter") parser = ArgumentParser("Colmap converter")
parser.add_argument("--no_gpu", action='store_true') parser.add_argument("--no_gpu", action='store_true')
parser.add_argument("--skip_matching", action='store_true')
parser.add_argument("--source_path", "-s", required=True, type=str) parser.add_argument("--source_path", "-s", required=True, type=str)
parser.add_argument("--camera", default="OPENCV", type=str) parser.add_argument("--camera", default="OPENCV", type=str)
parser.add_argument("--colmap_executable", default="", type=str) parser.add_argument("--colmap_executable", default="", type=str)
@ -26,29 +27,30 @@ colmap_command = '"{}"'.format(args.colmap_executable) if len(args.colmap_execut
magick_command = '"{}"'.format(args.magick_executable) if len(args.magick_executable) > 0 else "magick" magick_command = '"{}"'.format(args.magick_executable) if len(args.magick_executable) > 0 else "magick"
use_gpu = 1 if not args.no_gpu else 0 use_gpu = 1 if not args.no_gpu else 0
os.makedirs(args.source_path + "/distorted/sparse", exist_ok=True) if not args.skip_matching:
os.makedirs(args.source_path + "/distorted/sparse", exist_ok=True)
## Feature extraction ## Feature extraction
os.system(colmap_command + " feature_extractor "\ os.system(colmap_command + " feature_extractor "\
"--database_path " + args.source_path + "/distorted/database.db \ "--database_path " + args.source_path + "/distorted/database.db \
--image_path " + args.source_path + "/input \ --image_path " + args.source_path + "/input \
--ImageReader.single_camera 1 \ --ImageReader.single_camera 1 \
--ImageReader.camera_model " + args.camera + " \ --ImageReader.camera_model " + args.camera + " \
--SiftExtraction.use_gpu " + str(use_gpu)) --SiftExtraction.use_gpu " + str(use_gpu))
## Feature matching ## Feature matching
os.system(colmap_command + " exhaustive_matcher \ os.system(colmap_command + " exhaustive_matcher \
--database_path " + args.source_path + "/distorted/database.db \ --database_path " + args.source_path + "/distorted/database.db \
--SiftMatching.use_gpu " + str(use_gpu)) --SiftMatching.use_gpu " + str(use_gpu))
### Bundle adjustment ### Bundle adjustment
# The default Mapper tolerance is unnecessarily large, # The default Mapper tolerance is unnecessarily large,
# decreasing it speeds up bundle adjustment steps. # decreasing it speeds up bundle adjustment steps.
os.system(colmap_command + " mapper \ os.system(colmap_command + " mapper \
--database_path " + args.source_path + "/distorted/database.db \ --database_path " + args.source_path + "/distorted/database.db \
--image_path " + args.source_path + "/input \ --image_path " + args.source_path + "/input \
--output_path " + args.source_path + "/distorted/sparse \ --output_path " + args.source_path + "/distorted/sparse \
--Mapper.ba_global_function_tolerance=0.000001") --Mapper.ba_global_function_tolerance=0.000001")
### Image undistortion ### Image undistortion
## We need to undistort our images into ideal pinhole intrinsics. ## We need to undistort our images into ideal pinhole intrinsics.