Media Controller Project - Phase I

I recently began to work on a media controller project with two colleagues from ITP, Zeven Rodriguez and Michael Martinez-Campos. After much discussion, and our fair share of agreements and disagreements we have decided to develop an interface for music modulation. This device is going to be composed of a square horizontal surface coupled with a physical mouse-like object. By sliding the object across the surface, a user is able to modulate two attributes of the sound that is being generated.

Ideally, we would like to make the axis of this surface assignable (e.g. you could choose the effect/modulation associated to each axis). Also, it would be great if we could provide the user with the ability to play multiple sounds simultaneously, and to choose whether to control all sounds, or just a single sound with this surface.

That said, this project is being developed as part of our Introduction to Physical Computing curriculum. For our initial deadline 2-weeks from now we have decided to keep things simple; if we get things done sooner we may try to integrate some of these features.

To help get things done efficiently we have divided our roles and responsibilities. Zeven is taking the lead on creating the physical surface and object. Mike is working on investigating the solutions for the sound generation through Processing. I am leading the development of what I am calling the middleware - the application that gets the data from the sensors and feeds it to the program that will generate the music.

Creating the Connection (and a Virtual Grid/Surface)
Earlier today I finished the initial version of the processing application that will be responsible for reading data from the serial port, interpreting that data, and sending it onto to the sound generator. This is an important step in the evolution of this project, though I know that many updates will have to be made to this app during the coming weeks.

Pictures of Arduino Input for Test


One of the biggest challenges in creating this sketch was setting up the serial communication between the Arduino and Processing - we need to find a way to send data from two sensors to the computer. We decided to use the handshake protocol because it minimizes response delays.

Since I just learned how to use this type of protocol, it took me a little bit to get it working properly. Below I’ve included a brief overview of the issues I encountered along with the code I wrote for the Arduino, and a link to the Processing application.

Using the Handshake Communication Protocol
To set-up the handshake protocol, the first thing I did was to create a syntax for the communications. Unfortunately, that syntax did not work so I had to update it a few times to smooth out all of the kinks. Below I have shared the original and revised protocols.

• Initial protocol: “valueOne.valueTwo. \n\r” (e.g. “224.200.\n\r”). The new line and carriage return characters are appended by the println() function that I used to send data for the my first attempts.
• Final protocol: “valueOne valueTwo.” (e.g. “224 200.”). After numerous frustrating attempts to get the initial protocol to work, I decided to simplify the protocol as outlined above.

Another challenging issue that I encountered along the way was resolving the source of an “array out of bounds” error. After some investigation I realized that this error was generated within the function I created to read serial data. Upon further investigation I realized that I needed to confirm that a connection had been established with the Arduino before I starting to process the data that was coming in from the Arduino.

Once I understood the problem it was easy to fix. The solution was to add an “if” statement to check whether a piece data received by the computer is the first piece of data in the communication stream. I noticed that the code sample from this week’s labs features a similar solution.

Processing Sketch


Here is a link to the processing sketch that I developed (please note that I've commented out all serial communications related functionality in order for this sketch to run online). Below you will find the code for the Arduino.

Code for the Arduino

int analogPin1 = 0;
 int analogPin2 = 1;
 int analogValue1 = 0;
 int analogValue2 = 0;

 void setup()
 {
   // start serial port at 9600 bps:
   Serial.begin(9600);
   Serial.write('.');
 }

 void loop()
 {
   // read analog inputs:
  if (Serial.available() > 0) {

   analogValue1 = analogRead(analogPin1); 
   Serial.print(analogValue1, DEC);
   Serial.print(' ');

   analogValue2 = analogRead(analogPin2); 
   Serial.print(analogValue2, DEC);
   Serial.print('.');

   Serial.read();
  }
 }