This is a framework for generating incremental 3D scene graphs from data generated using the VirtualHome simulator.
This is based on SceneGraphFusion c++ framework. For the network part of the code, check here.
apt update
apt install git
# cmake
sudo apt install python3-pip
pip3 install cmake
# create ssh key and associate to your account in order to clone this project.
# * Generate new ssh key [link](https://docs.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent)
# * Adding a new ssh key to your account [link](https://docs.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account)
# Eigen3 3.3.90
# OpenCV 4.0.0
# This project will try to install them locally if they were not found.
loading meshes for online rendered view generation.
apt install libassimp-dev
With GUI:
# OpenGL & Glfw3
apt install freeglut3-dev libglfw3-dev
With graph prediction:
# Onnxruntime
# See https://github.com/microsoft/onnxruntime for more information
git clone --recursive -b v1.8.2 https://github.com/microsoft/onnxruntime
cd onnxruntime
# For Linux
./build.sh --config RelWithDebInfo --build_shared_lib --parallel
cd build/Linux/RelWithDebInfo
sudo make install
build the basic segmentation system
git clone {this_repo}
cd SceneGraphFusion
git submodule init
git submodule update
mkdir build
cd build
cmake ..
make
build with GUI or graph prediction, pass these options in cmake:
cmake -DBUILD_GRAPHPRED=ON -DBUILD_GUI=ON ..
./exe_GraphSLAM --pth_in path/to/3RScan/squence/
# or with GUI
./exe_GraphSLAM_GUI --pth_in path/to/3RScan/squence/
# to see usage and options
./exe_GraphSLAM --help
# to run with graph prediction (need to build with graph predict)
./exe_GraphSLAM_GUI --pth_in path/to/3RScan/squence/ --pth_model /path/to/the/traced/model/folder/
you can download the traced model of our pretrained network here. The model is trained with 20 NYUv2 object classes used in ScanNet benchmark, and with 8 support types of predicates.
In some platforms the CMake config for Assimp is called assimpConfig.cmake and
in others AssimpConfig.cmake. This mismatch can lead to CMake not finding the
library despite being available in the system. If this is the case, edit
cmake/UseAssimp.cmake to look like this:
find_package(assimp QUIET)
OPTION(WITH_ASSIMP "Build with Assimp support?" ${assimp_FOUND})
IF(WITH_ASSIMP)
MESSAGE(STATUS "WITH Assimp")
find_package(assimp REQUIRED)
INCLUDE_DIRECTORIES(${assimp_INCLUDE_DIRS})
ENDIF()
See assimp/assimp#3455, microsoft/vcpkg#14256 for more details