Interactive Concierge Wall Installation

Few things are as exciting as defeating the Episode 1 Floor 9 boss in Wolfenstein 3D for the first time. Having your boss give you a new project saying “We need to be able to interact with a wall in a bank’s lobby using gestures”  comes pretty close.

Initial prototypes of this client project involved tracking gestures via background subtraction using OpenCV and Processing, but we soon decided to explore the Kinect and leverage the depth data it provides. I went with the ofxKinect addon for openFrameworks for blob detection and gesture classification. Here’s how it turned out:

The wall itself comprised 12 Christie Microtiles, which are high resolution rear projection tiles capable of being stacked in any configuration. The content was mapped onto these tiles via the Watchout Multi-Display System by Dataton. The only thing left to do was bridge the communication and get the Kinect module talking to Watchout to trigger content updates.

However, Watchout is natively incapable of  talking to openFrameworks much in the same way that it is capable of talking to Adobe Flash. So we introduced a mediary Flash app to relay messages from oF to Watchout. The gesture data was sent over to Flash via a TCP link that I setup.

And thus it was that guests indulged the immersive novelty of a gesture driven interactive installation in the lobby of a bank.