The Design of Interaction with Music for Mobile Devices

Lately a large number of music applications have become available for portable phones and mobile gaming system. A large number of music-related applications have been released for these devices - including software for music listening, playing, producing and sequencing. These offerings range from virtual pianos, guitars, drums, harmonicas, sequencers and many other instruments to innovative interfaces that allow inexperienced users (or professionals) to create music in new ways.

iPhone Music Applications
As would be expected, most of the music-related applications for the iPhone provide listening or informational functionality. There are some pretty interesting applications in these genres (Pandora and Shazam being two of my favorites), however, the focus of this post is on mobile applications that enable users to create music, both directly and indirectly. So here are a few of my favorites:

BeatMaker
This is an example of applications that mimics existing music instruments. This one provides access to a drum machine and a sequencer. It has a pretty cool interfaces that works well for the most part. They've done a good job at integrating the sequencer and drum machine on to this small touchscreen interface. This application provides a decent set of features, some highlights include ability to loop sounds, to record and playback tracks, and to add new sounds for the drums. The drum is really easy to use featuring large virtual drum pads (the sequencer is a little less user friendly).

Priced at almost $20, this application is definitely not for everyone. It is easy to use considering the the functionality it offers. However, it is more complex than more abstract applications such as Ocarina or Bloom. You do not need experience with drum sets or sequencers to play with beatmaker - I had pretty close to none.



Pro Remote
This is the most high-end example of music interaction application on the iPhone. One distinction about this application is that it does not create music, it is a remote control for Pro-Tools running on a Mac. Nonetheless, it is quite impressive. The video below only shows one small feature of this application. Check out this other video, which highlights some other interfaces provided by this application.



Smule Ocarina
This is an example of a simpler but in some ways more innovative music interface designed specifically for the iPhone. Ocarina is an application that turns your iPhone into an wind instrument whose output can be broadcast to a worldwide network of users. The instrument itself is pretty interesting. You create sound by blowing on to the microphone, and you control it by using four different virtual buttons on the touchscreen display. The application is simple, designed to allow inexperienced to quickly and easily make sounds. That said I quickly grew bored of it.

The networking feature is pretty cool. You can broadcast your music continuously on to the network (even when you are logged off). You can also listen to the broadcast of others via a pretty cool interface that shows the location where the broadcast originates and includes a visual representation of the music - some element of this visual representation can even be customized by the person who makes the song.



Bloom
This application provides another example of a simple and innovative interface designed specifically for the iPhone. Similar to Ocarina, this app was designed to be easy to learn and fun to use for people who are not musicians. Created by Brian Eno and Peter Chilvers, this music instrument enables users to produce melodies simply by touching different parts of the touchscreen. The melodies progress in a loop-like fashion and they slowly melt away and disappear after 30 seconds or so. The instrument will also start playing when left idle for over 40 seconds or so. You can choose from a nine different moods to change the sound of the harmonies. Personally, I prefer this app to Ocarina - though it also grew old pretty quickly.



RJDJ Album
Unlike the other applications featured on this list, this one is primarily used for music consumption rather than creation, which is not to say that it does not create music. The concept behind this RJDJ application is that it generates music on-the-fly using live sounds that are captured in real time mashed up with pre-recorded elements. The live sounds surface in the music as samples, and influence the progression of other elements in the song (if you can call it that).

Based on RJDJ's website they are promoting a musical genre called "reactive", where the sounds that people hear are produced that very moment by digital devices. This is definitely a genre in its infance (I had the music world that will see further development as more artists experience in this




Nintendo DS

KORG DS-10 (w/ optional Straw TalkBox)
The iPhone (and other cell phones) are not the only mobile devices that have more advanced and innovative music applications. A few months ago Nintendo released an emulator for Korg's DS-10 synthesizer. This application turns a Nintendo DS into a mini Korg DS-10. Check out this video - it is definitely more advanced than anything I've seen for the iPhone.

Since I don't own a Nintendo DS-2 - I only have an iPhone - I wonder when an app like this will come out on the iPhone? Is it just a question of time? to what extent does the low-average prices of iPhone apps inhibit the release of software of this complexity?



Another cool thing about Korg DS-10 app on the Nintendo DS is that a hacker in Japan has created a simple way to turn this device into a talkbox, using a simple straw. Check out the video below, just beware because it is in Japanese.



For Nintendo DS there are also a number of open source homebrew music applications - here is 24 examples courtesy of synthopia. Most of these applications are for more experienced users (unlike Ocarina and Bloom on the iPhone). Most of them are designed to provide advanced features and to run these apps users have to learn how to install homebrew software on their Nintendo DS. This requires a higher level technical know-how that tends to be limited to heavier user of the device.

[some content sourced via PSFK]