Maestro

Maestro is a musical instrument, and an experiment in touch. Conducted like an orchestra and played by gesture alone, it is made for people with no musical training: a glove selects the instruments, a baton sets the tempo, and a wall of colored waves moves to the beat.

Because it is played in mid-air, the instrument gives the hands no physical feedback, none of the resistance a key or a string provides. The experiment adds haptic feedback, a small vibration in the glove and baton, and measures the difference it makes to how people play.

Conducting by gesture

The vocabulary is borrowed from conducting, and learned in minutes.

From gesture to sound

Both controllers are built from scratch. The glove reads pointing from a motion sensor and finger-bends from three flex sensors; the baton, 3D-printed, carries an accelerometer in its handle, set there rather than at the tip so it stays light and swings like a real one.

The projected waves are not decoration but a feedback channel of their own: the tempo drives their rhythm directly, one wave to every four beats, so what the player sees stays in step with what they hear.

What touch is worth

The vibration sits where a player would expect to feel the instrument, in the glove's palm and the baton, and marks what the hand would otherwise only see: it pulses as the level changes, and on every beat.

To learn whether it matters, the instrument was tested with eighteen participants, one group playing with the vibration and one without. Each set instrument levels with the glove and held a tempo with the baton, measured on how well they played and how much they enjoyed it.

The vibration made the instrument measurably easier to use, but no more enjoyable to play. The two are independent, and an instrument has to be designed for both.

Made with Andrea De Carlo, Lucrezia Di Bari, and Pietro Cau.

The full report, with the method, results, and references, is below.