CAM WOW!

Published: The Interactive Graffiti Project
Hands-Free Interaction with the Kinect

For the “closed” opening night of the new Contemporary Art Museum (CAM) in Raleigh, co-developing a Kinect-based interactive art project with David Rieder, Patrick Fitzgerald and Lee Cherry.

Presented at the (Closed) Opening of the Contemporary Art Museum (CAM) in Raleigh
Friday, 29 April 2011

Project developed by:

Summary of project:

The “Interactive Graffiti Project” was developed collaboratively by faculty in Art + Design and English at NC State University. It is the first of several projects planned by the group, based on their work with the Microsoft Kinect.

The project is a proof of prototype for a next generation of digital humanities projects based on the capabilities of and data from the Microsoft Kinect. The Kinect is part of a shift in computer design based on natural-user interfaces (NUIs). The “Interactive Graffiti Project” proved to us that the Kinect can be a platform for robust project development.

The Flash project projected on the wall during the pre-opening of CAM Raleigh enabled performers to “write” complex graphics and text on one of ten different “stages” zooming across the scene. Once the Kinect “calibrates” a user, it streams data in the form of x-y-z coordinates through a tcp server to a Flash application. That data is then used as the basis for user-interactions with the visual assets on the screen.

Patrick Fitzgerald conceived of the project, guided its development as our project lead, and designed all of the visual assets for it, which included complex, multi-layered movie clips, short videos, text, and line drawings.

Bishon Bopanna rewrote the “stick man” program in the Prime Sense/NITE open source initiative so that x-y-z data comprising a user’s position and movements streamed to an ActionScript3 class for the Flash project. Related to this, he created a TCP server, which allows Flash to get the C++ data from the Kinect through node.js.

David Rieder developed the user registration program in ActionScript 3, which parses, sorts and prioritizes user data streaming from the Kinect sensor from a socket connection to the node-based TCP server.

Lee Cherry developed the ActionScript 3 “engine” that translated the x-y-z user data to the visual assets dynamically moving and transforming on the screen. He also developed a sound visualization that represented the constantly-changing levels of ambient sound in the performance space.

Video:

Click image to see video
Photographs:

Photo taken by Lee Cherry

Photo by Lee Cherry

Photo by Lee Cherry

Photo by Lee Cherry

Photo by Lee Cherry

Tic Tac Grow; Autism Spectrum Disorder

An Autism Living and Learning Community

Working with a small start-up firm to refine their mission and business model as they transition to a non-profit technology company to offer new platforms for social growth, learning and development. Developing a vision that children with autism and their families each have voice and the opportunity to grow and develop their own social networks in our increasingly global society.

Project Development: Lee Cherry, David Franks (US Animation & Sales, RoboRocket), Jodi Hubble

Zoom Raleigh @ Fayetteville Street

Zoom Raleigh, as a 3D interactive installation, allows viewers to interact with digital media related to the history of the arts, architecture and culture of Raleigh, NC. Viewers can manipulate the digital model by rotating the city itself, selecting buildings of interest and interacting with narrated slideshows of historical photographs.

Project Team: Lee Cherry, Melissa Church, Pat FitzGerald, Stacie McGowan

Zoom Raleigh is a rich interactive interface that has been installed in the window of the Urban Design Center right on Fayetteville Street. Further refining and developing 3D interfaces for exploring urban spaces. This project is the culmination of that work along with the refinement of large touch-screen interfaces.

Zoom Raleigh has incorporated multi-media montages, archival materials, historic photos, and interviews with Raleigh leaders like Greg Hatem. The result is a massive interactive screen that sits behind glass that entices passers-by to reach out and play with the interface. Part art, and part technical feat, the work both educates the public and reinforces the technology culture of Raleigh.

http://vimeo.com/6552428

http://bcove.me/angnfnh8

Zoom Raleigh as a 3D interactive digital installation allows viewers to interact with digital media related to the history of hte arts, architecture and culture of Raleigh, NC. Featuring an interactive 3D visualization of downtown Raleigh, viewers can manipulate the digital model by rotating the city itself, selecting buildings of interest and interacting with narrated slideshows of historical photographs and text specific to selected sites.

Exploiting experimental touch and hand gesture recognition technology developed by students and staff a the Advanced Media Lab at NC State University, College of Design, this system utilizes camera recognition, back screen projection and large-scale store-front windows as display screens. Zoom Raleigh represents a new generation in inexpensive human/computer interaction platforms that can be operated in interior/exterior urban environments.

Zoom Raleigh is co-sponsored by the Raleigh City Museum, City of Raleigh, Raleigh Arts Council, American Image Graphics, and Empire Properties.

Design Camp

[ UPDATE: Follow Design Camp  2009 ]

Hosted a series of Animation Workshops in conjunction with Design Camp and the Contemporary Art Museum. This was the first time that a digital animation component was introduced into the Design Camp disciplines and we hope that it will be a sure hit as an additional feature in the future.

[youtube xtNucvoChzs]

We had a fantastic time trashing the lab and creating some exciting stop motion animations as well as a quick stop action short film. We featured non-stop sessions of storyboards and concept sketches, fruit-loop fights, mock-ups and enthusiastic budding directors. With a lot of help from the event organizers, student TAs, camp counselors and featured returning faculty member Ryan DeWitt we were able to do a 5-day workshop and showcase 80 student animations and 5 short film animations on the final Saturday Design Camp Reception. A big thanks goes out to Nicole Welch and Pablo del Valle for helping us organize the events, keeping us on track and stocking us up with supplies!

Not to be outdone, Santiago Piedrafita – Chair of the Department of Graphic Design at NC State University College of Design introduced future design students to the art of Typography and projected light. Emphasis was placed on the process of exploring a visual composition in studies of scale, layering, opposition and tension using graphic pattern, icons and typography. After projecting it at a large scale it allowed students an opportunity to see how layering, juxtaposition and scale change the graphic form and meaning.

The next session for Design Camp is the day camp series starting July 7th and then a final round of overnight Design Campers during the week of July 20th. We are finalizing the gallery and tutorial mini-website so check it out later this month at 3by3.wordpress.com

For more information about Design Camp check out cam.ncsu.edu!

[youtube mJyGUiHHGV8]

Consumer Culture Garden

The Consumer Culture Garden
eatcollaborative.com

December 2007 – February 2008

Developed an interactive art installation utilizing Flash AS2, infrared camera tracking, intelligent agents and flocking routines. Working on museum design, installation fabrication and experience marketing as a unique and artistic endeavor to increase public awareness of branding, product placement and marketing.

[ New Britain Museum of American Art ]

[ Lush Life: Grand Graffiti ]

The team that comprises EAT formed in June 2004 in response to on-going debate around the subject of aesthetics of virtual environments and a localized epidemic of low-grade ennui.

[flv:/portfolio/consumerculturegarden/eat.flv 400 300]

Development for The Consumer Culture Garden began in June 2004, and continued until about a week before the opening at The North Carolina Museum of Art. The latest version is now available for viewing in the NEW/NOW exhibit at the New Britain Museum of American Art.

The process was based on the Eames design process, reflecting an iterative exploration into a wide range of solutions for every aspect of the installation.The growing sophistication of virtual environments has changed the way we communicate, work together, and entertain ourselves. New technologies allow us to tour buildings yet to be built and play games in simulated, but surprisingly realistic, environments. For the purposes of education, entertainment, communication, commerce and novelty, these media technologies can now convincingly simulate the natural world.

-

Non-contact Interactive Displays

Large-scale, Interactive Digital Signage
Moving away from MakingThings/Teleos hardware and going straight Flash 9 w/modified hardware. We include standard button interaction, drag+sweep gesture recognition, multi-touch and ability to scale to almost any window dimension w/o the need for depth or rear-view tracking…

The Interactive Window is a back projected web-cam based system that can essentially turn any glass window (of any size) into a touch screen interface. This is a “non-contact interactive display” designed to be operated by intuitive human hand gesture. Gestures of press, push, scale and rotate are all programmable interactions of this technology. The system has the potential to be multi-touch (two handed gestures simultaneously recognized) and initial attempts at doing this have been successful.

Similar technologies are extremely expensive, have their own closed operating system and require multiple cameras. This platform is based on open source culture and is developed with Adobe Flash 3.0 development application – a widely available and popular development software, making the technology accessible to most (non-programming) designers and content developers. Additionally, this technology is based on projected light, so it easily scaleable and customizable in its size and proportions. Set up cost is extremely inexpensive by comparison, and thus, potentially very popular to the general public.

This updated “Interactive Mirror” presents a lower cost of entry into the interactive space and leverages open-source hardware/software to allow non-programmers and designers to develop digital spaces. We are not so much concerned with pixel-point precision and touch/pressure sensitivity… We are more interested in non-contact, gesture based interactive systems for information displays and exhibits that are easy to implement and open to designers.

Technician Article – “Touch”

[flv:/portfolio/interactiveMirror02.flv 350 200]

[flv:/portfolio/interactiveMirror01.flv 350 200]

Sources of inspiration:

• Microsoft.com/surface
• Perceptivepixel.com
• Displax.com
• Inwindowoutdoor.com
• Adobe Interactive Window
• Infiniti Interactive Mirrors

CAM|Contemporary Art Museum

CAM|Contemporary Art Museum
Contemporary Art Foundation, Raleigh
http://cam.ncsu.edu/
Spring 2007 (ongoing)

Experimental research into alternative, interactive new media installations for education and entertainment. This form of large-scale digital exhibit, introduces designers to sensor data from external sources that utilize dynamic content creating a unique interactive experience that is both larger-than-life size and immersive in the experience. Utilizes Flash AS2, gesture recognition, infrared web camera tracking, Makingthings Teleos system and XML/RSS datasets.

[ IMI: Interactive Mirror Installation ]

This is experimental project into alternative, interactive new media installations for education and entertainment. “CAM,” no longer a “Contemporary Art Museum,” but keeping the acronym, seeks to get community involvement in artistic & creative events and hopes to infuse creativity into education in downtown Raleigh. It is not our intent with this project to turn designers into engineers. Our intention was based on the principle of having non-technologists work together with technology outside their expertise to produce artifacts that are informative, entertaining and promote an exchange of dialogue over design, technology and society. At the completion of this project typical design students in this research collaboration developed an understanding of technical design integration, a vocabulary of the field and a working knowledge on the application of technology in different disciplines. The initial large-scale, Interactive Mirror Installation, introduced designers to sensor data from external sources and utilized dynamic content creating a unique interactive experience.

Design Expo Raleigh – DXR.07

Design Expo Raleigh 2007 – Design On
http://dxraleigh.com/
November 2007

The purpose of Design Expo Raleigh ’07 is to present great design to the public and to give designers, both professionals and students, a forum to display and discuss their work. A unique showcase of applied arts coming together as a whole, Design Expo Raleigh promotes broader awareness of many different design disciplines and creates an opportunity for local designers to network with each other, students, and the public.

This two-week event combined fund raising, project management, event coordination, marketing/promotion and website integration. Design Expo is the brain-child of Donald Corey (Other Edge) and was developed in conjunction with Michael Bustin (MeldUSA), Erick Mehlman (Cline Design), Amanda Robertson (iMediaGraphics), Jessie Schaefer (Signal Design) and a host of volunteers.

[youtube 7MNos62_T9U]

-

World Children’s Festival

The World Children’s Festival
http://www.icaf.org
Summer 2007

Developed FlicFlix, A Flash animation applet which integrates live video and real time green screen chromakey technology to fully immerse students, characters and props into the animation world.

[ FlicFlix, Animation Composition ]
[ Digital Comics for Literacy ]

The most important international event for children, their teachers and parents in the world, the World Children’s Festival is hosted by the ICAF every four years, traditionally on the National Mall in Washington, DC. The World Festival integrates the arts, science, sport and technology to create a complete synesthetic experience — a total work of art that transforms the National Mall. Leading educators and experts come together to heal, inspire and unify children and peoples across the globe.

The Festival becomes a turning point in the participants’ lives, a touchstone to guide them in the future. The 2007 World Children’s Festival took place in Washington, DC on June 22-26. From June 23 to 25, the World Festival occupied from the 4th to the 7th Streets of the National Mall.

-

collaboration: designTech + REU

DISCOURSE
Design Tech, an REU site funded by the National Science Foundation (NSF) Research Experiences for Undergraduates (REU) and the Department of Defense (DoD) Awards to Stimulate and Support Undergraduate Research Experiences (ASSURE), is an interactive visual design hothouse with projects spanning artificial intelligence, graphics, visualization, and visual design. Projects include cinematic camera control for interactive narratives and games; automated tours through virtual, visualized environments; agent-based visualizations of streaming data such as RSS feeds and sensor telemetries; and PDA-based navigation tools.

The Design Tech undergraduate experience involves a 12-week summer research program that provides outstanding research opportunities in the areas covered by Design Tech faculty research interests. Students receive a stipend, travel and living expenses during their participation in Design Tech. This Research Expereinces for Undergraduates (REU) program is supported by the National Science Foundation and the Department of Defense.

The Design Tech Summer 2006 program ran from May to August, 2006. The targeted audience consists of both male and female students that have attained at least junior status. Undergraduate participants receive an stipend, an allowance for travel to/from Raleigh, North Carolina, housing and meals. Students will be housed in University Housing in campus dormitories. These rooms are furnished and are within walking distance of all the research laboratories. For more information about applying visit designtech.ncsu.edu.

TECHNICAL ASPECTS

(RSS) feeds are used to quickly see a summarized list of all recent content on a website. The main objective of this year’s DesignTech program is  to visually depict these feeds using Processing, an open source programming language used to easily create interactive graphics, much like Adobe’s Flash tool. For our project, we chose to build a tool that allowed users to monitor feeds in the background while performing other tasks.

We have created several prototypes varying information displayed as  well as visual designs, using several elements to identify the source, age and value of the information being displayed. Some information sources  include: Amazon.com,  Google News, CNN, BBC News and Craigslist. The several visual designs we  have experimented with include 2D designs using floating text “rain drops,” ticker tapes, as well as 3D designs using spheres and z-axis positioning.

RSS (Really Simple Syndication) feeds are a tool on the Internet that notifies users about changes in websites such as online newspapers, directories, or blogs. The goal of this summer’s DesignTech program is to visually represent RSS feeds using the interactive design language, Processing 1.0. The main objective of this project is to represent feeds from online stores and exchanges, such as amazon, ebay, and craigslist.

We constructed a virtual store or gallery using images and information from the feeds to create an environment for users to browse the products. Since not all feeds contain images the current prototype finds photographs in the online image directory flickr. Eventually the prototype will depict feeds from  ebay, because this site includes links to photographs in its feeds. The application imagery is a hallway created by images on the walls that move past the viewer. These images represent theproducts that the user is “shopping” for. The user can then change categories and receive a new set of  images and feeds.

COLLABORATIVE EFFORT
DesignTech ’06 was an opportunity that provided outstanding research initiativees for Computer Science and Art+Design Students. Working together they develop innovative solutions and novel applications of internet technology for analyzing large, dynamic datasets. It provided an open forum to discuss industry trends and social implications of privacy, information and computing.