Abstract:
The goal of this project is to explore the role of haptics, proprioception, and kinesthesia in
playing musical instruments. This project will examine the importance of tactile sensory
feedback used when playing a simulated conga drum (high and low notes) with and without the
presence of a contact surface. To create the virtual interface, a Microsoft XBOX 360 Kinect will
be used to track the user's hand movement and produce a MIDI percussion note corresponding
to the motion of the user's respective hand (left/right hand, high/low note) upon contacting the
inferred or physical drum surface. The experiment will test the ability of the user to reproduce
a sequence of notes at the appropriate velocities (integer 0 to 127 scale) with auditory feedback
and with and without haptic feedback.
Materials:
Timeline:
| April 23 |
Parse Kinect data (hand positions in x and y and hand velocity in z) |
| April 30 |
MIDI mapping to gestures (position corresponds to note value, hand velocity corresponds to note velocity, aftertouch, modulation, etc.) |
May 7 |
MIDI output from Kinect application (COM port/tcp for MIDI signals) |
| May14 |
Virtual environment (clean UI for Kinect), control MIDI library () |
| May 21 |
Compare haptic feedback to non haptic feedback |
| May 28 |
Fine tuning |
| June 4 |
Presentation, writeup, and |
