Multi-Agent Robotic System using ROS2 & Isaac Sim

Advanced multi-agent robotics platform featuring autonomous navigation, manipulation, and coordination

🎥 View Demo • 📖 Documentation • 🚀 Quick Start

---

📝 Overview

This repository contains a comprehensive multi-agent robotic system implementation using ROS2 and NVIDIA Isaac Sim. The system includes various robotic agents including mobile manipulators, forklifts, and Spot robots, with integrated behavior trees, path planning, and simulation capabilities.

🎥 Demo Video

Click the badge above to download and view the demo video.

Alternative viewing options:

• 🎬 View on Instagram

🎯 Project Overview

This system demonstrates advanced robotics concepts including:

• Multi-Agent Coordination: Multiple robots working together in shared environment
• Behavior Trees: Hierarchical decision-making for complex robot behaviors
• Real-Time Planning: Dynamic path planning with obstacle avoidance
• GPU-Accelerated Perception: AprilTag detection using NVIDIA Isaac ROS
• Industry-Standard Tools: ROS2, Isaac Sim, MoveIt2, Nav2

Supported Robots

Robot	Capabilities	Use Case
🦾 Mobile Manipulator (Nova Carter + Jaco)	Navigation + Manipulation	Warehouse automation, pick-and-place
🚜 Forklift	Autonomous navigation, pallet handling	Material transport
🐕 Boston Dynamics Spot	Quadrupedal locomotion, inspection	Complex terrain navigation

Robot Visualizations

📊 Project Highlights

• ✅ 3 Different Robot Platforms integrated into single system
• ✅ GPU-Accelerated Computer Vision using NVIDIA Isaac ROS
• ✅ Behavior Tree Architecture for scalable robot control
• ✅ Real-Time Path Planning with Nav2 stack
• ✅ 6DOF Manipulation with MoveIt2 integration
• ✅ Reinforcement Learning locomotion controllers (policy files included)

📑 Table of Contents

• Quick Start
• Project Structure
• Supported Robots
• Usage
• Development
• Configuration
• Troubleshooting
• License
• Author

🚀 Quick Start

Prerequisites

• Ubuntu 22.04 LTS
• ROS2 Humble
• NVIDIA Isaac Sim 4.5+
• CUDA-compatible GPU
• Python 3.10+

Installation

1. Clone the repository with submodules:

   git clone --recurse-submodules https://github.com/YousefSamm/Multi-Agent-Robotics-System-using-ROS2-Isaac-Sim.git
   cd Multi-Agent-Robotics-System-using-ROS2-Isaac-Sim

2. If you already cloned without submodules, initialize them:

   git submodule update --init --recursive

3. Build the workspace:

   colcon build --symlink-install
   source install/setup.bash

📁 Project Structure

Core Components

• noham_bt/ - Behavior tree implementations for robotic agents
• noham_path_planner/ - Navigation and path planning modules
• noham_custom_interfaces/ - Custom ROS2 message and action definitions
• noham_localization/ - Localization and mapping components
• noham_map_server/ - Map server for navigation
• noham_apriltags/ - AprilTag detection and pose estimation
• cmdvel_to_ackermann/ - Velocity command conversion utilities
• spot_controller/ - Boston Dynamics Spot robot controller
• isaacsim/ - Isaac Sim simulation configurations and scripts

Dependencies (Git Submodules)

• apriltag_ros/ - AprilTag ROS package (christianrauch/apriltag_ros)
• isaac_ros_nitros/ - NVIDIA Isaac ROS NITROS bridge (NVIDIA-ISAAC-ROS/isaac_ros_nitros)
• isaac_ros_apriltag/ - Isaac ROS AprilTag detection (NVIDIA-ISAAC-ROS/isaac_ros_apriltag)
• isaac_ros_common/ - Isaac ROS common utilities (NVIDIA-ISAAC-ROS/isaac_ros_common)
• magic_enum/ - C++ enum utilities (Neargye/magic_enum)

Additional Packages

• kinova_moveit_config/ - MoveIt2 configuration for Kinova Jaco arm (used with mobile manipulator)

🤖 Supported Robots

Mobile Manipulator

• Robot: Nova Carter Base + Jaco arm
• Capabilities: Navigation, manipulation, object detection through apriltags
• Launch:

  ros2 launch isaacsim run_noham_mobile_manipulator.launch.py

• Note: For the mobile manipulator, you must also launch the MoveIt2 controllers through the kinova_moveit_config package

Forklift

• Robot: Custom forklift model
• Capabilities: Autonomous navigation, pallet manipulation
• Launch:

  ros2 launch isaacsim run_noham_forklift.launch.py

Spot Robot

• Robot: Boston Dynamics Spot
• Capabilities: Quadrupedal locomotion, inspection
• Launch:

  ros2 launch isaacsim run_noham_spot.launch.py

🎮 Usage

Running Simulations

1. Launch specific robot:

   # Mobile manipulator
   ros2 launch isaacsim run_noham_mobile_manipulator.launch.py
   
   # IMPORTANT: For mobile manipulator, also launch MoveIt2 controllers:
   ros2 launch kinova_moveit_config noham_jaco.launch.py
   
   # Forklift
   ros2 launch isaacsim run_noham_forklift.launch.py
   
   # Spot robot
   ros2 launch isaacsim run_noham_spot.launch.py

Navigation and Path Planning

2. Launch navigation stack:

   # Mobile manipulator navigation
   ros2 launch noham_path_planner mobile_manipulator_path_planner.launch.py
   
   # Forklift navigation
   ros2 launch noham_path_planner forklift_path_planner.launch.py
   
   # Spot robot navigation
   ros2 launch noham_path_planner spot_path_planner.launch.py

Behavior Tree Operations

3. Run behavior trees:

   # Mobile manipulator behavior tree
   ros2 launch noham_bt mobile_manipulator_bt.launch.py
   
   # Forklift mission
   ros2 launch noham_bt forklift_mission.launch.py
   
   # Spot behavior tree
   ros2 launch noham_bt spot_bt.launch.py
   
   # Room navigation (simple demo)
   ros2 launch noham_bt room_navigation.launch.py

🛠️ Development

Adding New Robots

1. Create robot-specific packages in the workspace
2. Add launch files for simulation and control
3. Update behavior trees if needed
4. Configure path planning parameters

Updating Dependencies

To update submodules to their latest versions:

git submodule update --remote
git add .
git commit -m "Updated submodules to latest versions"

Building Individual Packages

# Build specific package
colcon build --packages-select <package_name>

# Build with specific dependencies
colcon build --packages-up-to <package_name>

📋 Configuration

Isaac Sim Settings

• Simulation configurations are located in isaacsim/config/
• USD files for different robot models are in respective directories
• Policy files for reinforcement learning are in isaacsim/policy/

Navigation Parameters

• Configuration files are in noham_path_planner/config/
• Behavior tree configurations are in noham_bt/config/

🐛 Troubleshooting

Common Issues

1. Submodule initialization failed:

   git submodule update --init --recursive --force

2. Build errors:

   # Clean and rebuild
   rm -rf build install log
   colcon build --symlink-install

3. Isaac Sim connection issues:
   • Ensure Isaac Sim is running
   • Check ROS2 environment variables
   • Verify network connectivity

Getting Help

• Check individual package README files for specific documentation
• Review launch file parameters for customization options
• Examine log files in the log/ directory after building
• Create an issue in this repository for bugs or questions

📄 License

This project is licensed under the MIT License. Individual submodules may have their own licenses.

👤 Author

Yousef Sammour
Mechatronics Engineering Graduate | Automation & Robotics Specialist

• 📧 Email: yusef.sammour@gmail.com
• 💼 LinkedIn: Yousef Sammour
• 🐙 GitHub: @YousefSamm
• 📍 Based in Homs, Syria

---

Developed as graduation project for Bachelor of Mechatronic Engineering, Homs University (2025)

⭐ If you find this project useful, please consider giving it a star!

---

Note: This repository uses Git submodules to manage external dependencies. Always clone with --recurse-submodules or initialize submodules after cloning to ensure all dependencies are available.