This tutorial is no longer being updated. For latest version go to:

Oak-D Stereo Camera | docs.fictionlab.pl

Home

>

Components

Oak-D Stereo Camera

This tutorial will guide you through the process of connecting an Oak-D camera to your Leo Rover.

Prerequisites

No items found.

Home

>

Components

Oak-D Stereo Camera

This tutorial will guide you through the process of connecting an Oak-D camera to your Leo Rover.

OAK-D is the ultimate camera for robotic vision that perceives the world like a human by combining stereo depth camera and high-resolution color camera with an on-device Neural Network inferencing and Computer Vision capabilities. It uses USB-C for both power and USB3 connectivity. In robotics, stereo cameras are commonly used for mapping the environment, object recognition, 3D scanning and in many other.

What to expect?

After completing this tutorial, you will have successfully connected an Oak-D stereo camera to the Leo Rover in ROS, and you will possess basic knowledge about the use of stereo cameras.

Prerequisites

No items found.

List of components

  • Oak-D Series Stereo Camera
  • Any mounting solution (eg. Universal Camera Mast)
  • USB-A to USB-C wire
  • (optional) USB-C to DC power supply.

Mechanical integration

There are multiple use cases for stereo cameras. Since we cannot predict them all, we cannot design a perfect mount for all of them. That's why we provide a Universal Camera Mast that can fit a wide range of projects.

Wiring and electronics connection

Whether you are using our Universal Camera Mast or your own custom solution, you'll need to connect the camera to the rover somehow.

The USB port placed on the mounting plate of the rover does not support USB 3.0 that's necessary to run the camera. That's why you'll need to connect the USB-C wire straight to the USB ports of the rovers built-in Raspberry Pi.

Kacper from our team likes to remove the micro-usb port entirely and run the wire straight through the newly created hole. This solution, combined with the use of a cable gland, keeps the MEB box waterproof.

Second common solution involves printing a new cover for the MEB box. Dev cover found here, provides a wide opening for running wires straight into the MEB. This solution can be extremely useful in projects that don't need to be waterproof.

If the camera isn't the only thing connected to the rover you might need to provide an external power connection.  Powerbox might come in handy in such situations.

Software integration

Installing the packages

Add USB rules to your system

Install depthai-ros packages(only available in ROS noetic)

Adding camera model

As we want to have the camera model visible with the rover model, we need to change robot.urdf.xacro file in /etc/ros/urdf directory. Package depthai_descriptions is installed alongside depthai-ros and it contains Oak camera models and xacro macros that we can easily add.

To include the Oak camera macro add following lines to /etc/ros/urdf/robot.urdf.xacro file.

This macro contains a lot of parameters that are set to a default Oak-D Pro camera setup on a rover. You can adjust those parameters to your needs (e.g. if you have the camera mounted in a different position or you are using a different camera model).

In the end, your robot.urdf.xacro file should look like this:

/etc/ros/urdf/robot.urdf.xacro

Creating launch and configuration files

To launch the camera you can, create your own launch file, use one of the example packages from depthai-ros package or you can use the provided launch file which provides basic functionalities such as camera image and pointcloud created from depth data.

In /etc/ros create a .yaml configuration file. This example contains configuration for Oak-D Pro pointcloud mapping.

/etc/ros/leo_oak_pcl.yaml

Next, also in /etc/ros create a launch file that will use the previously added configuration and will create a nodelet manager and appropriate nodelets. Nodelets are used for communication efficiency.

/etc/ros/leo_oak_pcl.launch

For more information about oak configuration and usage examples you can check out the depthai-ros github repository.

Launching camera nodes on system startup

Include your launch file in the robot.launch file, so that your node will start at boot.

In /etc/ros/robot.launch add this line between <launch> tags:

/etc/ros/robot.launch

Now your Oak-D camera ROS nodes will start at launch.

Examples

Reading and visualizing the data

The topics from camera node should be now visible. You can check them with rostopic tool - topics from your node will have prefix specified by you in the camera ros argument.

If you have ROS installed on your computer, you can get the camera image with rqt, and see the robot model with camera attached using rviz.

Before starting any of the programs, make sure you completed the Connecting other computer to ROS network section of ROS Development tutorial:

ROS Development

Now you can start rqt, and from plugins -> visualization chose image view and then chose your topic.

Don't worry if the quality is low, and the image is lagging. It is due to transfer of the data through your rover and computer.

You can also start rviz, and add robot model in the display view. Chose also base_link as fixed frame, and you should see the rover model with camera attached.

TODO: add rviz photo after robot model is complete

You can also display tf tree in rqt, to see if everything is configured correctly (plugins -> visualization -> TF Tree)

What next?

Stereo cameras are commonly used in projects involving autonomous navigation, you might be interested in a tutorial about it. They are however, not the only way of teaching a Leo Rover how to move on it's own. Check out our line follower tutorial if you want to learn more. You can also check our Knowledge Base for more instructions.

Contents

Need help? Contact us - contact@fictionlab.pl