Ghost Cubes

Overview

Ghost Cubes is an interactive piece that explores the concept of applying physical objects to visualize music through color/shape movements and using the body as the controller without touching any buttons or knobs. The project adapts leap motion, p5.js, and Arduino.

See the code: https://github.com/bowenbowen/ghost-cubes

Team

You Zhang, Jinpei Wei, Xiaofang Fan

Timespan

4 weeks, 2019

Role

Coding (primary responsibility), concept development, material preparation, assembly

Inspiration

Light, music, and body

Launchpads and MPCs are common music controllers, which have color LED buttons placed in a cubic grid. What if we use such a cubic form to visualize the music instead of controlling?

To work around this idea, we did some research and came across the idea of using Japanese origami ghost cube to be the core mechanical structure because of its flexibility and Leap Motion as the technology choice to support touchless interaction.

Making

Origami mechanism, and diffusion

After making the smaller scale model of the cubes, we understood the mechanical structure better. Thus, we were able to sketch out the whole object to combine the cubes and electronics. To diffuse the LED light, we used tracing paper to create the cubes.

Hardware and software

Leap Motion, Arduino, and p5.js

How it works

We use a Leap Motion Controller to capture the parameters around hand movements, which can be accessed in p5.js through the JavaScript client. After processing these variables in p5.js, we send them to the Arduino side through serial communication. The Arduino board then takes care of controlling the behavior of the hardware.

Hardware

We used 9 LEDs from the Adafruit NeoPixel LED Strip, 1 Arduino servo motor, 9 tracing paper folded cubes, and a couple of couple vinyl pieces made from the laser cutting.

Sound manipulation

Using the filter() function from p5.js, we mapped the hand rotation and height values as sound volume and frequency to create filters that change the music while playing. The output turned out pretty well without much tuning.