OpenNI Application "Bouncing Ball2"

in Japanese

Introduction

In the previous page, I implemented an application with the use of the OpenNI library. The application includes the simple simulation by the Newtonian dynamics. Recently, I have replaced the simulation with the Open Dynamics Engine(ODE). In this page, I give you an explanation on the new application. (Here is my first introduction to the ODE.)

Demo

I start with a demo movie.

Unfortunately, the image quality has degraded after uploading to this blog.

Development Environment

1. Mac OS X 10.8.2
2. Processor：3.06 GHz Intel Core 2 Duo
3. Memory：4GB
4. Xcode4.5.1

Primary Libraries

1. boost-1.51.0
2. pcl-1.6.0
3. OpenNI-Bin-Dev-MacOSX-v1.5.4.0
4. ode-0.12

My Source Code

Here is my source code. I do not confirm the behavior of the application on other environments.

Compile Options

Header Search Path:

Path	Purpose
/usr/include/ni/	to include the OpenNI headers
/opt/local/include/	to include the Boost and ODE headers
/usr/local/include/pcl-1.6/	to include the PCL headers
/opt/local/include/eigen3/	to include the Eigen header on which the PCL depends
/opt/local/include/vtk-5.10/	to include the VTK headers on which the PCL depends

Library Search Path:

Path	Purpose
/opt/local/lib	to link the Boost libraries
/usr/local/lib	to link with the PCL libraries

Other Linker Flags:

-lboost_thread-mt -lboost_filesystem-mt -lboost_unit_test_framework-mt -lboost_chrono-mt -lboost_program_options-mt -lboost_system-mt /usr/lib/libOpenNI.dylib -framework OpenGL -framework GLUT -lpcl_filters -lpcl_kdtree -lpcl_segmentation -lpcl_surface -lpcl_search -lpcl_features -lpcl_common -lode -lpcl_io

Usage

An execution file name is BouncingBallWithOde. Running it with an argument --help, it writes the following usage statements on the standard output. Running it without any arguments, it opens a viewer and begins to display frames captured by the Xtion Pro Live. Typing "m" on the keyboard under the situation where the application focuses on the viewer, a triangular mesh is constructed from a point cloud. The point cloud is calculated from the depth data in the frame at the moment when typing "m". Typing "b" generates virtual balls in the captured scenes, and the interactions between the balls and the objects in the environment are drawn in real time. You can generate balls as many times as needed. Though an error occurs in the 29th line, no major disruptions are observed. To quit the application, type "q."

Brief Overview of Source code

View Point of OpenGL

The view point of the OpenGL is as follow: A unit of length in the application is mm. You might need some changes to the arguments in the 6th line according to the environment captured by your depth sensor.

Transforming Depth into Point Cloud

A depth data is transformed into a point cloud in the method Device::create_cloud. In this application, two point clouds are generated, one (cloud_) is for displaying the color frames and another (cloud_xyz_) is for calculating a triangular mesh. RGB values are set in the 25,26th lines. The depth data is transformed into the point cloud in the 29th-30th lines. The point cloud used to calculate the triangular mesh is generated in the 33th line.

Drawing Color Image

The point cloud cloud_ is passed to the method Viewer::display_point_cloud as follows: The arguments buf and size indicate the pointer to the head address of cloud_ and the size of cloud_, respectively.

Calculation of Triangular Mesh

In the method Device::convert_to_polyton_mesh, the triangular mesh is constructed from the point cloud cloud_xyz_ by the use of the PCL library. (Here is my first introduction to the PCL library.) You might need to change parameters in the 27th-36th lines according to your environment.

Making Balls

A ball treated in the application is an ODE object. It consists of a dBody object and a dGeom one. The former manages the rigid body dynamics and the latter manages the collision detection. dSphere is the derived class from the base class dGeom. In the method PhysicalSimulator::create_balls(), balls are registered to the ODE framework. world_ in the 20th line and space_ in the 41th line are the dWorld and the dSpace objects, respectively. In the ODE framework, the object which moves based on the rigid body dynamics has to be registered to the dWorld object. The object which collides with other objects has to be registered to the dSpace object.

Registration of Triangular Mesh

The triangular mesh is registered to the ODE framework in the method PhysicalSimulator::register_triangular_mesh. triangular_mesh_ is the dGeomID object.

Collision Detection

The ODE library supports the method dCollide which handles the collision detection. The collision detection is carried out per frame. In the loop, the various properties are set to the collision points. The variable bounce in the 14th line corresponds to the reflection coefficient.