https://vimeo.com/95045553

What started as a project that would reconstruct point cloud data through Kinect Fusion SDK along with PCL had its limitations that led me to take a detour to getting Time Varying Datasets and finally landing in the Kinect Fusion SDK.

There was a very steep initial learning curve with respect to setting up the drivers and the software. My Macbook was not supporting the Kinect drivers as it had a low end graphics card and I had to use Dr.Ponto’s Alienware laptop with an NVIDIA GTX780 graphics card that was pretty fast. Compiling PCL and its dependencies, OpenNI and PrimeSense was the next step which had a few issues of their own while interacting with Windows drivers. These initial phase was very frustrating as I have not really coded much in the Windows OS and had to figure out how to setup the hardware, drivers and software. It was almost mid-March when I had the entire setup running without crashing in the middle.

Although it was a late start, once the drivers and the software were setup, it was all exciting and fast. I was able to obtain datasets automatically using the OpenNI Grabber interface. I had to just specify the time interval between successive captures and the program saved them as PCD files (Color and Depth). It wasn’t late until I was able to get 400 PCDs of a candle burning down with PCDs captured 1 second apart from each other which would give a realistic 3D rendering of the scenario. The viewer program was pretty similar that took in the number of PCD files and the time interval between the display as arguments and visualized these 3D datasets.

Further on, I also tried learning the Windows SDK that is provided along with the Kinect. The Kinect Fusion basics is a beautifully written piece of code that obtains PLYs when scanned with the Kinect. PCL offers options to convert these into PCDs which was the desired final format. I also tried running multiple Kinects simultaneously to obtain data that would fill in the shadow points of one Kinect but I was not able to debug an error while Windows SDK’s Multi Static Cameras option. Given more time or as a future work, I believe using multiple Kinects to obtain PCD files would be a good area to explore. Working on these obtained PCD files like hole filling and reconstruction would also be good topics to cover in the future.

Here is a comparison of the image that is used in pointclouds.org(the one I put up in my second post as a target) against the image that I have obtained. Both are screenshots of PLYs.

Left: Mesh from pointclouds.org; Right: Mesh obtained by me

The datasets can be found at:

Candle: https://uwmadison.box.com/s/j1zheh8b46fbxjs079xs

Walking Person: https://uwmadison.box.com/s/lxkr7a7io5rbz84xy4uw

Office: https://uwmadison.box.com/s/8mfccacpewkptymicx67

All in all, I am happy with the progress of the work. If the drivers were not a big hindrance, I would have had a better start in the beginning of the semester. Nevertheless, it was a great learning experience and an interesting area of study.

Here is the office dataset with 8 PCDs (and their original PLYs) obtained from Kinect Fusion SDK.

https://uwmadison.box.com/s/8mfccacpewkptymicx67

I have tried to combine Kinect Fusion and the Time-Varying Dataset concept into one by obtaining two time varying PLYs of a scenario and converting them into PCDs. The link to this small dataset could be found at

https://uwmadison.box.com/s/bgw63fi54y5ir5wkedav

Meanwhile, since I did not have OpenNI support in my machine as I have deleted it while installating Kinect SDKs, I could not try running the program to visualise it. Nevertheless, the comparison of the two PLYs looks so in MeshLab.

The scans look intact and the Kinect’s depth sensors were actually working much better compared to last week since I moved the Kinect slowly across the space.

The codes to the capture and viewing program that I have worked with until now can be found at

https://github.com/nsubramaniam/rekinstruct

For some reason, using the Multi Static Kinects option in the Kinect SDKs which obtains points from two Kinects gives an error while opening the PLY file it saves. The error corresponds to something like “Header has an EoF” which I believe is erroneous metadata. I am looking into it. Will update once I get to know something.!

There has been a bit of a change in my approach to obtain PCDs. I have moved on to obtain PCDs(PLY, to be precise) using the Kinect Fusion Colour Basics. It scans the volume for 3D points and allows me to save in one of the three formats (OBJ, STD and PLY). I chose the PLY format as I can easily convert PLYs to PCDs using PCL.

Here is a video of me scanning around the room to obtain the mesh.

http://vimeo.com/93961128

The mesh finally looks like this in MeshLab.

The next steps would be to take a few PCDs and probably use the Time Varying PCD program to visualise them like a movie.

Meanwhile, there happened to be a few issues with the datasets in the file locker. So, I have uploaded the files in UW-Madison Box. Here are the links to the datasets.

Candle Dataset 1 second : https://uwmadison.box.com/s/j1zheh8b46fbxjs079xs

Walking Person : https://uwmadison.box.com/s/lxkr7a7io5rbz84xy4uw

I have uploaded the candle dataset on the file locker system. You can download it from here.

https://filelocker.discovery.wisc.edu/public_download?shareId=1d35394ba86c9251776af95cb2821f46

Password: ReKinStruct

These are 40 PCD files of snapshots taken of two candles with a 10 second interval between consecutive shots.

I have also uploaded a video of the ReKinStruct viewer for the candle dataset here. The video however is like a fast forward version of candles burning where the 40 PCD files are displayed back to back with a time interval of 1 second. This makes a close-to-reality rendering of candles burning but only 10x faster.

http://youtu.be/zwA9J8xv248

I believe I kept the candles too close to the Kinect and didn’t get it’s depth data perfectly. However, the shadows and the candle melting show how the time varying datasets look in 3D.

I have been trying to setup two Kinects to grab data simultaneously and have been getting the classic segmentation fault.

I wonder if it goes to back to the original issue where the OpenNI grabber was not sensing the Kinect. I have looked online for some tutorials and most people seem to have used two OpenNI grabbers simultaneously. Will dig in about this a little more and post progress.!