Kinect for impact

So what to do, if you have a new product and it is not quite ready for a big show, but you want to get people excited about it and want them to try it out and look at it freely by themselves. This was the problem posed to us here at Identitymine recently along with our good friends at Critical Mass, for their client Nissan. I was lucky enough to be a part of this project and thought I would share some of our experiences.

The Chicago autoshow was due in a matter of weeks and the concept car wasn't quite ready for the show, but they had to find a way to show off the new car and impress people with their new offering for the year. The solution was to push technology further than before and try something a little different. So with help from Critical Mass we produced a 3D model and placed it in a gesture based environment that utilizes Microsoft Kinect. The results were stunning and a real testament to how far technology and interactive experiences have come.
This is in my opinion is a vision of the future of virtual showrooms.

Interactive platform where user walks around car. The large monitor is hanging
from the ceiling in the top right corner.

So what exactly did it take for us to pull this off? Firstly the 3D model was several million polygons, and had an immense amount of detail from not only the model itself but also the texturing. The car was always intended to be a real time 3D object not pre-rendered, as you normally see on websites and in Flash. In order to make this work, some extra time was required to reduce the poly count and reproduce some of the texturing to make it a reasonable object to play with without needing a super computer to calculate all the information points. Even with this effort, due to the fact we had to retain so much detail because the car was meant to be reflective of the real car, the machine we used was very high end and ran a powerful graphics card, but it was an off the self HP machine and is a consumer model. The end result was quite simply amazing to see a real time 3D render of a such a detailed and large model. Think Forza 4!!!

Million Polygon Model with HDR lighting effects

The next challenge was to build the lighting and environment, this was originally built out of real 3d objects and it was intended to show the car in a real setting and lighting scene, but the decision was to reduce the environmental clutter to allow the car the shine as the star. Although the environment was well within capabilities of the team and machine. Using HDR maps we produced a rather nice lighting schema that was very much inline with real car photography that uses huge light boxes. This really brought out the curves and reflection throughout the vehicle. Of course the great thing about a 3D object and environment is everything was editable and even possible to see results in real time. After lighting and general model tweaking everything had to become an interactive experience. This was a real discovery process.

Two Kinects Interaction Zones

It was always imagined that you should be able to walk around the car as you might at a dealership. This of course was not quite possible with a static TV and single Kinect sensor. So the next effort was spent making the Kinect track the user in front of the screen and rotate the car as you walk from side to side, amazingly the Kinect would even track you up and down motion as you walked across side to side, giving a real feeling of you being in front of the car. Some issues occurred with the "visible zone" of the Kinect and this was resolved with a second sensor increasing the area it could see, but this also require deeper development to make it work. With more time and effort we might have been able to work out a way to project a more curved screen experience, but even so, the car would rotate from front to back within the distance from one side of the screen to the other. One of the amazing features of the demo was the ability to open the doors and go inside the car to check out the seating and dashboard. The nice part was not only the incredible texturing and lighting detail but also the fact that using zones in front of the Kinect sensor we were able to allow you to "step back" to any of the 3 rows of seating within the car as well as glance around and change seating location.

High Res Interior, with Drivers Eye View

The end result was truly amazing and has been gaining a lot of attention at the Chicago Autoshow. All of this was acheived in weeks not months or years. An amazing effort from our talented crew here at Identitymine and Critical Mass. The main interaction was performed in front of a 52" TV and the large projection was placed above the car for all to view. An amazing thing to see at the autoshow. Overall the project had some interesting challenges and pushed the current technology really hard but the results speak for themselves, we have come along way, and I believe this shows a glimpse of what is now possible. The car model and interactions are stunning and proof that we are close to very virtual showrooms where you can pick cars and see them in almost life like settings.

If you are near Chicago or attending the show, I really recommend you check it out.