Leap Conductor

Leap Conductor logo

Inspiration

We wanted to know what a professional conductor could feel when directing a band but it ended up being much more..

What it does

It lets you create music using different notes and instruments by moving your hands on the air and making some special gestures like pointing with the finger or closing your fist. This tool can also be used to remix music, letting you accelerate or decelerate the rhythm of the song and adding special effect to the main track.

How we built it

We use a library called MIDI.js, a pretty old but powerful library written in plain JS. With this library we were able to reproduce MIDI files in real time in the browser. The leap-motion gesture and position tracking is done by our node app which communicates with the browser using Socket.io.

Challenges we ran into

Identify hand gestures reliably, making all this things in a browser might be a bit unstable -> trying to avoid unnecessary processing was also a handicap. Frontend and backend communication was also a bit challenging. Synchronizing tracks, sound and gestures while still being able to change the 'Tempo' in real time was hard.

Accomplishments that we're proud of

We are proud of the final result. At first we didn't think there would be so many interesting and funny possibilities but it ended up being a very funny to use and very user friendly app to create and mix music.