Vision-Based Human-Following Robot
A mobile robot that detects a person, follows them at a safe distance, avoids obstacles, and can switch modes using hand gestures.
- Build a robot that can stay with a moving person without colliding with the environment.
- Detect the person with vision, estimate distance from depth, follow with a simple controller, and fuse LiDAR for obstacle avoidance. Gestures toggle modes.
- Working person-following on a Triton robot with gesture-based mode switching, teleoperation, and autonomous modes.
How it works
Detect
Find the person in the camera view.
Follow
Keep a safe distance while moving.
Avoid
Use LiDAR to steer around obstacles.
Switch
Hand gestures change modes.
- Camera
- Detect
- Distance
- Follow
Motors
LiDARfeeds into Follow
In action
What I worked on
- Obstacle-avoidance with LiDAR fusion and safety checks
- Laptop-to-Triton bridge for remote control, telemetry, and logging
- Hand-gesture recognition wired into the control state machine
Technical details
YOLOv5 on a Jetson Nano with an Intel RealSense D435 RGB-D camera; LiDAR for collision avoidance; MediaPipe for gesture inputs. The stack supports both teleoperation and autonomous following. Project notes described recognition under varied lighting and occlusion, following at walking speeds with small distance error, gesture switching with low latency, and collision-free obstacle avoidance in store-like layouts.