Introduction
I implemented an application which
- generates a point cloud from a depth data captured by the Xtion Pro Live,
- calculates a normal vector for each point in the point cloud using the Point Cloud Library(PCL),
- introduces virtual balls into an environment on the viewer,
- and simulates collisions between the balls and the objects in the environment.
The application does not retrieve normal vectors from each frame, but from a certain frame just one time.
The simulation is executed in real time under those normal vectors.
I used the OpenGL library to draw frames.
Development Environment
- Mac OS X 10.7.4
- Processor：3.06 GHz Intel Core 2 Duo
- Memory：4GB
- Xcode4.3.3
Primary Libraries
- Boost-1.50.0
- PCL-1.6.0
- OpenNI-Bin-Dev-MacOSX-v1.5.4.0
- Sensor-Bin-MacOSX-v5.1.2.1
Source Codes
Here is my source codes. I confirmed that the application works well on my machine.
Compile Options
Header Search Path:
| Path |
Purpose |
| /usr/include/ni/ |
to include the OpenNI headers |
| /opt/local/include/ |
to include the Boost headers |
| /usr/local/include/pcl-1.6/ |
to include the PCL headers |
| /opt/local/include/eigen3/ |
to include the Eigen headers on which the PCL depends |
| /opt/local/include/vtk-5.10/ |
to include the VTK headers on which the PCL depends |
Library Search Path:
| Path |
Purpose |
| /opt/local/lib |
to link with the Boost libraries |
| /usr/local/lib |
to link with the PCL libraries |
In addition, to use the OpenGL, I set "
-framework OpenGL." More detailed information is in my source codes.
Usage
The executable file name is
OpenNiViewer. Running it without any arguments, it prints the following statements.
Running it with setting the path to a file "SamplesConfig.xml" to
--config, it opens a viewer for a video captured by the Xtion Pro Live. The file "SamplesConfig.xml" is enclosed in my source codes. I don't verify the behavior of the application with values other than default ones for
--width and
--height.
Typing "p" on the keyboard under the situation where the application focuses on the viewer, the derivation of normal vectors begins.
After seeing "Normals has been extracted" on the standard output, typing "i" yields virtual balls in the captured scenes, and the interactions between the balls and the objects in the environment are drawn in real time.
To quit the application, type the escape key.
Brief Overview of Source Codes
View Point of OpenGL
The view point of the OpenGL is as follows:
A unit of length in the application is mm.
You need to change the argument in the 8th line according to your environment.
Transforming Depth into Point Cloud
A depth data is transformed into a point cloud in the method
OpenNiDevice::create_cloud.
I generate two point clouds, one (
cloud_) is for displaying the color video and another (
cloud_xyz_) is for calculating normal vectors.
RGB values are set in the 26th, 27th, and 28th lines. The depth data is transformed into the point cloud in the 33th line.
The point cloud for calculating normal vectors is created in the 34th line.
Drawing Color Image
The point cloud used for the color video,
cloud_, is passed to the OpenGL method
glInterleavedArrays as
Derivation of Normal Vectors
The cloud used for deriving normal vectors (
cloud_xyz_) is processed in the method
OpenNiDevice::calculate_normals.
The method
initialize in the 6th line is as follows:
The region of the local nearest search is set in the 9th line. This argument varies depending on your environment. The plane is calculated using points inside the region. The normal vector of the plane is what we want.
The normal vectors is derived with each typing "p" on the keyboard.
If you change position of the Xtion Pro Live, you have to type "p" again.
Configuration of Balls
The virtual balls is registered in the method
OpenNiViewer::register_balls.
The radius of the ball is set in the 25th line. The initial position, the initial velocity, and the acceleration are set in the 31th, 32th, and 33th lines, respectively.
These values also vary depending on your environment.
Configuration of Walls
A wall is placed at the front of the viewer to avoid the balls disappearing from view.
Search of Normal Vectors
For each frame, the normal vectors located adjacent to the balls are detected by means of the kd-tree.
kdtree_ in the 8th line is the object of the class
pcl::KdTreeFLANN<pcl::PointXYZ>. Because the object returns
nan if it fails to detect neighbors, the returned value is checked in the 12th line.
