Sympathetic Vibrations | AUMI-zing Software Aims to Give Individuals of All Abilities the Power to Jam

To an outsider, the scene at the Sound + Vision studio of the Lawrence Public Library in Lawrence, Kansas, on Tuesday may have looked strange – an enthusiastic group of individuals, some of whom are differently abled, waving their hands and moving their heads to create a panoply of musical sounds.

But to Kip Haaheim and Abbey Dvorak, professors at the University of Kansas, the music represents a groundbreaking advance in technology and furthers their aim of providing a more inclusive outlet for musical expression.

Haaheim and Dvorak are both researchers in AUMI-KU InterArts, a department at KU that has devoted much of the last decade to seeing how many uses it can find for its subject software – AUMI, which stands for “Adaptive Use Musical Instrument.”

The software was initially developed 10 years ago by renowned composer Pauline Oliveros. The current version, which is available on desktop and iOS (an Android version is in the works), is fairly simple, using the camera from a computer or smart device to track the motion of its user and guide a colored cursor over onscreen triggers called “sound boxes.” As the cursor moves over any of the boxes, a sound is played. And each box plays a different note.

Dvorak, who is an assistant professor in Music Education & Music Therapy, said the motivation behind the software allows it to be a valuable music therapy tool.

“I like how the instrument was first created, which was so people with limited voluntary movement could participate in an autonomous way,” Dvorak said. “So you can be independent users and create music with others, forming a community. And I really like that inclusive practice.”

There are an almost limitless number of sound box configurations, colors and instruments to explore. The boxes can be configured to play in differing keys on the pentatonic scale and the user can tap on screen to designate which part of their body controls the cursor.

It all works as advertised, though the instrument can be difficult to play with any level of precision. So nobody will be shredding Stevie Ray Vaughn-esque solos on the software any time soon.

For Haaheim, who is a director of the Music Theory & Music Composition Department at KU, the real strength in AUMI is that it can be used by virtually anyone to make music. Without the need for extensive training or the ability to make complicated finger patterns on an instrument, the AUMI team at KU has been able to join children and individuals suffering from profound disabilities with more experienced musicians.

“One of the interesting things about the video tracking is that it’s an equalizer in a way,” Haaheim said. “If you come in and you are already a highly trained musician, you’re not necessarily going to be any better at playing the instrument than somebody who has never seen it before.”

Performers use AUMI on iPads to participate in an improvised jam session at Sound + Vision Studio in the Lawrence Public Library. The software allows individuals of varying abilities to play music together using motion capture to trigger synthetic instruments. (Dan Calderon | Flatland)

Performers use AUMI on iPads to participate in an improvised jam session at Sound + Vision Studio in the Lawrence Public Library. The software allows individuals of varying abilities to play music together using motion capture to trigger synthetic instruments. (Dan Calderon | Flatland)

Performers use AUMI on iPads to participate in an improvised jam session at Sound + Vision Studio in the Lawrence Public Library. The software allows individuals of varying abilities to play music together using motion capture to trigger synthetic instruments. (Dan Calderon | Flatland)

These performances involving both trained and untrained musicians were born out of the department’s desire to get more community involvement and test the software with a variety of participants. The result is the monthly AUMI Jam Session series, which is in its second year, thanks largely to the department’s partnership with the Lawrence Public Library and Independence Inc. – a Lawrence-based nonprofit aimed at providing resources for individuals living with disabilities.

Tuesday represented the second AUMI Jam: Battle of the Bands and included four participants of varying musical skill and ability, who improvised a few minutes of music using the software. As with all AUMI Jams, the resulting track was mixed on the spot and immediately uploaded to a Sound + Vision Playlist on SoundCloud.

The fruit of Tuesday’s performance, “Storm of Friends,” was heard and critiqued by Bonnie McLarty and Ben Justis, graduate students in KU’s music composition program.

The work KU’s AUMI team has put into creating these diverse performances has not gone unnoticed. In January, the department was awarded a $35,000 grant from the National Endowment for the Arts, which it hopes to use to keep its jam sessions going and continue researching how the software can benefit users.

And for Dvorak, those benefits are numerous.

“As we do more research using this software with people of all abilities, we will better understand people’s responses to music. And their responses to inclusive practice,” Dvorak said. “A lot of times, when the group is finished, they listen to the track and process it. What sounds did they like? What did they notice about their music? And part of that processing is determining how they feel about the music. And you get a sense that people are very proud of what they’ve done. And so there’s a sense of accomplishment, pride and community.”

— Dan Calderon is Kansas City native, an attorney, and contributor to Flatland. You can contact him by emailing pdancalderon@gmail.com, or on Twitter @dansascity.