Efficient 3D Scene Graph Update based on Scene Change Detection

This repository is the code for the Efficient Scene Graph Update. EfficientSGU updates 3D scene graph that enable a robot agents to understand dynamic environment.

To Do List

✅ Release source code about Scene Graph Generation

✅ Release Dynamic Scene Graph Update

Installation

EfficientSGU has been tested with Python 3.10, Torch 2.0.1 & CUDA=11.8.

Setup

NOTE: Sometimes certain versions of ubuntu/windows, python, pytorch and cuda may not work well together. Unfortunately this means you may need to do some trial and error to get everything working. We have included the versions of the packages we used on our machines, which ran Ubuntu 20.04.

To create your python environment.

# conda environment
conda create -n EfficientGraph python=3.10
conda activate EfficientGraph

# Install some setting
## to your own setup, If you have another setting.
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia

# Install the Faiss library for quick indexing of pointclouds
conda install -c pytorch faiss-cpu=1.7.4 mkl=2021 blas=1.0=mkl

# Install Pytorch3D
conda install https://anaconda.org/pytorch3d/pytorch3d/0.7.4/download/linux-64/pytorch3d-0.7.4-py310_cu118_pyt201.tar.bz2

# Install cuda toolkit.
# to your own setup, If you have another setting.
conda install -c conda-forge cudatoolkit-dev

# Intall the others
pip install tyro open_clip_torch wandb h5py openai hydra-core distinctipy ultralytics dill supervision open3d imageio natsort kornia rerun-sdk pyliblzfse pypng git+https://github.com/ultralytics/CLIP.git

# Check your Cuda path
export CUDA_HOME=/your_path/envs/conceptgraph

#Install EfficientSGU: Whenever you want to install EfficientSGU
cd /your_path
git clone https://github.com/AutoCompSysLab/EfficientSGU.git
cd EfficientSGU
git checkout ali-dev
pip install -e .

Datasets

Comming Soon!

Setting up your configuration

We usee the hydra package to manage the configuration. Jist edit ther entries in the corresponding .yaml file in EfficientGraph/hydra_configs. main config is rerun_realtime_mapping.yaml file

defaults:
  - base
  - base_mapping
  - replica
  - sam
  - classes
  - logging_level
  - _self_

force_detection: !!bool False
save_detections: !!bool True

use_rerun: !!bool True
save_rerun: !!bool True

paths

The first thing to set in your config files is where you've installed conceptgraphs and where your data is. Update this in the ./EfficientGraph/hydra_configs/base_paths.yaml file. For me, it is:

repo_root: /home/your_repos/concept-graphs
data_root: /home/your_data_root

Run

To generate initial scene graph: initial_SG_generation.py To update scene graph: final_SG_updaste.py

# set your folder
cd /EfficientSGU/EfficientGraph/slam
python initial_SG_generation.py
pyrhon final_SG_updaste.py