Below is a brief description of the hardware and the system. More in-depth information can be found in technical report.
Hardware
The hardware setup that has been used is the following
- CVL's mobile robot:
RC car with mounted laptop, camera and webcam
4 core Intel Core i7 CPU + Nvidia 480M GPU
Point Grey USB camera
System overview
The functionality of the robot can be divided into a learning phase and a following phase. When the robot is started it will be stationary and learn the appearance of the person in front of it. After finishing the learning phase the robot will then start to follow the learnt-in person.
The image-capture module are responsible for collecting images from the camera. These images are then accessed by the other modules.
The person detector an image as input to detect possible candidates for the operator. These candidates are then compared to the learnt appearance of the operator. In case of a good enough match the size and position of the operator are then mapped to a relative distance and angle which is input to the steering system.
The steering system calculates suitable steering and throttle signals to the engines with respect to minimizing the relative angle and deviation from a reference distance.
The Obstacle Detection takes input from the mounted web-cam and checks for obstacles in the robots path. If an obstacle is found this overrides the effort to follow the operator and the robot is stopped until the obstacle is removed.
The image-capture module are responsible for collecting images from the camera. These images are then accessed by the other modules.
The person detector an image as input to detect possible candidates for the operator. These candidates are then compared to the learnt appearance of the operator. In case of a good enough match the size and position of the operator are then mapped to a relative distance and angle which is input to the steering system.
The steering system calculates suitable steering and throttle signals to the engines with respect to minimizing the relative angle and deviation from a reference distance.
The Obstacle Detection takes input from the mounted web-cam and checks for obstacles in the robots path. If an obstacle is found this overrides the effort to follow the operator and the robot is stopped until the obstacle is removed.
Graphical user interface
In order to be able to follow the robots decision process a friendly graphical user interface (GUI) has been develoeped. The GUI concist of three different images, the eye cameras vision, the web cameras image with detected filtered edges applied in blue and the warning zone. Obstacles in the warning zone will be colored completely blue if they are visible but below the amount of obstacle pixels threshold. If the obstacle has reached above the threshold, the obstacle will be colored red. Besides the images, there is also an information area which displays current values for speed, distance to operator and etc. The image below shows the actual interface.
The systems GUI
1. The eye camera image | 2. The web camera image | 3. The warning zone | 4. Information area