Dual-Mode Control System for a Robotic Rover

A 4-wheel drive robotic rover controllable via two intuitive, wireless modes: a wearable inertial gyro-glove and a vision-based hand gesture system with a live First-Person View (FPV) video stream. This project is built on a Raspberry Pi 5 and leverages an ESP32 for the gyro controller and MediaPipe on a host laptop for gesture recognition.

!

🌟 Features

• Dual-Mode Control:
  • [cite_start]Inertial Mode (Bluetooth): A wearable glove using an ESP32 and MPU6050 sensor translates hand tilts directly into robot motion. [cite: 49]
  • [cite_start]Vision Mode (Wi-Fi): A host laptop uses its webcam, OpenCV, and Google's MediaPipe to recognize hand gestures (by counting fingers) to control the robot. [cite: 49]
• First-Person View (FPV): A live video stream from the Raspberry Pi's camera is displayed on the laptop, allowing the operator to see from the robot's perspective.
• Wireless Communication: Uses both Bluetooth for the simple gyro mode and Wi-Fi (TCP Sockets) for the more complex vision mode.

---

🛠️ Hardware Components

Component	Role in Project
Raspberry Pi 5 (8GB)	Main computer on the rover; runs servers, controls motors, and streams video. [cite: 77]
Pi Camera Module 3	The "eyes" of the robot for the FPV stream. [cite: 106]
L298N Motor Driver (x2)	High-power bridges between the Pi's GPIO pins and the DC motors. [cite: 96]
4WD Robot Chassis	[cite_start]The body of the robot, including four TT DC gear motors. [cite: 96]
ESP32 Dev Module	[cite_start]The brain of the wearable gyro-glove; reads sensor data and sends commands. [cite: 135]
MPU6050 Gyro/Accelerometer	[cite_start]The inertial sensor in the glove that detects hand tilts. [cite: 137]
Power Bank	Power source for the Raspberry Pi.
Separate Motor Battery	[cite_start]A 7.4V LiPo battery is recommended for reliable motor performance. [cite: 97]
Laptop with Webcam	Runs the client for vision-based control and displays the FPV stream.

---

🔌 Wiring & Connections

Raspberry Pi to L298N Motor Drivers

This setup uses two L298N modules, one for the left wheels and one for the right.

Start (Raspberry Pi)	End (L298N)	Purpose
GPIO 17	IN1	Left Motor Forward
GPIO 18	IN2	Left Motor Backward
GPIO 22	IN3	Right Motor Forward
GPIO 23	IN4	Right Motor Backward
5V Power Pin	5V / VCC	Powers the L298N's logic chip
Ground Pin	GND	Critical: Creates a common ground reference
(Motor Battery)	12V / VMS	Positive (+) from 7.4V Motor Battery
(Motor Battery)	GND	Negative (-) from 7.4V Motor Battery

ESP32 to MPU6050 Gyro Sensor

This diagram shows the four wires needed for the gyro glove.

Start (ESP32)	End (MPU6050)	Purpose
3V3	VCC	Critical: Powers the sensor with 3.3V
GND	GND	Common Ground
Pin D22	SCL	I2C Clock Signal
Pin D21	SDA	I2C Data Signal

---

💿 Software & Setup

1. ESP32 (Gyro Glove)

• Open the ESP32 code in the Arduino IDE.
• Install the Adafruit MPU6050 and BluetoothSerial libraries from the Library Manager.
• Upload the sketch to your ESP32 board.

2. Raspberry Pi (Rover)

• [cite_start]Ensure your Pi is running Raspberry Pi OS. [cite: 80]
• For FPV mode, install the camera library: sudo apt update && sudo apt install -y python3-picamera2.
• The gpiozero, serial, and socket libraries are pre-installed with standard Python.

3. Laptop (Control Station)

• Create a Python virtual environment to keep dependencies clean.
• Install the required libraries:

  pip install opencv-python mediapipe

---

▶️ How to Run

This project has two distinct modes.

Mode 1: Vision-Based FPV Control (Wi-Fi)

1. Connect to Wi-Fi: Ensure both your laptop and Raspberry Pi are on the same Wi-Fi network.
2. Get Pi's IP Address: On the Pi's terminal, run hostname -I.
3. Run the Server on the Pi:

   python3 pi_fpv_server.py

4. Run the Client on the Laptop: Update the PI_IP_ADDRESS in the laptop script, then run:

   python laptop_fpv_client.py

   Two windows will pop up: one for your hand gestures and one for the FPV stream.

Mode 2: Gyro Glove Control (Bluetooth)

1. Pair Devices: Use bluetoothctl on the Pi to pair with your ESP32_Glove.
2. Bind the Connection: Create the virtual serial port on the Pi (this must be done after every reboot):

   sudo rfcomm bind 0 YOUR_ESP32_MAC_ADDRESS

3. Run the Receiver on the Pi:

   python3 gyro_receiver_simple.py

   The Pi will now listen for commands from the glove.