Esophageal Sphincter

Collab with Clare Carroll and Tim Sherman

The prompt for our project was to recreate an exterior space in the blackbox theater. We chose to explore the unseen space of the digestive tract in our piece. We combined mouth and colon to create an other-worldly internal experience.

This project involved getting many different pieces of technology to communicate with each other. We wanted the Kinect to control elements of both the audio and the video, so the openFrameworks patch had to receive and process the kinect data, and then send data out to two different locations. For the video, we sent a particle simulation image out to Millumin by using Syphon, which was implemented into the OF patch. For the audio, we sent OSC messages containing the coordinates of the blobs which the kinect sensor was tracking to the Max patch responsible for outputting the Ambisonic audio. In Millumin, we overlaid the particles patch image with our found and pre-recorded videos, and sent it all to a projector hung on the ceiling. This projector projected onto a mirror, hung at an angle such that the projection was thrown onto the floor, where the audience viewed it.

With the Kinect, we looked for people or “blobs” around the projection. For each blob, we established an attraction point for the OF particle system. This allowed the virus-like particles to swarm toward audience members as they walked around the experience.

The audio is comprised of three main tracks. The first is pulled audio from a colonoscopy procedure; the original audio was cut using iZotope RX 5 Audio Editor and then edited with reverb in Ableton so that only the doctor’s voice could be heard. The second is a recording of swallowing integrated with granular feedback and delay. The last track is comprised of improvised mouth sounds (slurping, chewing, etc.) with reverb and granular synthesis. The swallow and mouth sound tracks were recorded with a Yeti mic.

The sounds were then processed with Max using the High Order Ambisonic (HOA) Library decoder and automation map. The doctor’s voice is automated to travel around the room, the swallow sits in the middle of the room, and the mouth sounds are controlled by OSC data from the Kinect camera. You can listen to the original audio, remixed audio, and the combined audio here.