Cats vs. Bat Game - My Final ICM Final

Now that finals are finished I finally have some time to devote to this journal. Needless to say, a lot has happened during the last month that I have not yet shared here. Let's begin with an update regarding the ICM final.

Over the weekend I finished my final project for ICM, which is an updated version of the fangs invaders game that is titled "Cats vs Bats". This updated version of the game is almost unrecognizable from the previous one. Here is a link to an online version of the game (please note that you need to have a fast internet connection to play it because it is not web optimized), followed by a brief description of the game coupled with a list of the latest updates.

You can also download the game here: mac version | windows version.

Project overview: cats vs bats is simple space invader-like game. In this game, a small oriental shorthair cat named Sasha is our only hope to save the world against a swarm of diseased bats.

Though currently this project is screen-based only, I am working on developing a cat toy that will be used in conjunction with this application. A single joystick controller has been created that can be used to control either or both of these projects (so that you can play the game and with your cat at the same time). Also, the proximity of the cat to the toy will enable a player to earn bonus points in the Cats vs. Bats game.

Main updates: Here is an overview of the main updates that I have been working on over the past several weeks. Over the winter break I plan on delving deeper into a few of these areas (such as use of vectors to drive bat motion).
• Improve game scoring logic and add score to game play environment.
• Create an online high-score database.
• Improve animation by making flight patter more random and natural.
• Update look and feel of the game by adding back images and improving design.
• Include bonus points opportunities.
• Add stages with increasing levels of difficulty.

High Score List - Finally Solved

Over the past week I have been grappling with a simple problem: determining how to create a high score list for the Fangs Invader game. This seemingly simple problem gave me a complete mental block that took a week to thaw. I am happy to report that on Friday night I was finally able to solve this problem. Here is a simple sketch that displays the functionality that I created followed by an overview of my solution.

The Architecture
From an application architecture perspective, I decided to do most of the heavy lifting in Processing because I am know it better than PHP. Hence the PHP script only has two functions: (1) saving the high scores to a text file; (2) displaying the high scores from the text file. Processing takes care of the following functions: (1) Determining if current player’s score qualifies him to be added to the high score list; (2) capturing the player’s name, if he has a high score; (3) re-sorting the high score list to include the new score and name.

PHP Script
Two weeks ago I struggled with PHP to create a script that was able to read data from a query string and save that data to a text file, and read data from the same text file and display it appropriately. After much trial and error I successfully put together a short script that is able to accomplish this feat.

I found PHP to be a very temperamental language. I am not used to working with un-typed languages – I am not skilled enough to take advantage of the additional flexibility they provide while I seem to get caught up in many unknown and unexpected quirks and behaviors. Nonetheless, I have not been dissuaded from learning this language and hope to build my skills over the next three semesters.

Processing Sketch
Shortly after I finished the PHP script I started working on the Processing sketch. In no time I had the Processing application reading from and writing to the PHP script. I quickly lost steam when I tackled the sorting of the highscore list. This is not to say that this problem is particularly hard, however, in the road to solving it I encountered a major a blank.

At first the problem seemed simple, I created a short algorithm that seemed to work fine. I was actually very satisfied with myself for having solved this problem so quickly and simply. I had even started to integrate the solution into my fangs game.

So my solution did work fine unless there was more than one entry on the list with the same score. When this occurred the algorithm would select the name of one entry and apply that name to all others with the same score. Another issue was that the logic for the Fangs high score list is reversed - the better scores are the lower scores. Therefore, I needed to create an algorithm that is able to locate the lowest score that is above zero.

This is when I hit into a hard brick wall, head-on. The problem was back and I did not know where to even start. My attempts at writing pseudocode were totally unhelpful at first. My mind was so focused on integrating the functionality into the game and doing other things to the game that it just did not want to deal with this problem anymore.

In retrospect, I see how my initial approaches to this problem were overly complex (I won’t even try to explain it here). At first I had a hard refocusing and “seeing the forest from the trees” so that I could tackle the problem from a different perspective. It took me 6 days to figure out a different approach to solve this problem. Part of this process required that I stop working on this problem for a few days so that my mind could disassociate from it.

Here is a pseudocode-like overview of the solution that I developed for sorting the highs core list. All of the functionality outlined below is encapsulated in the changeList() function that is part of a class called PHPconnect. To support my algorithm I declared three array variables, each one holds eleven variables: a position for each from the current high score list and one position for the current player’s score.

The newLocation array is used to store the new location of each element on the top score list with references to their current location on the scoreList array, which holds the pre-sorted rank of each element on the old top ten list along with the new player’s score, saved in the eleventh position in the array. All elements in the newLocation array are initialized to -1 when the changeList() function is called. This is relevant because the newLocation array plays an important role in processing lists that feature multiple players with the same score.

The tempScore and tempName variables are used to temporarily hold the re-sorted top score and top name lists. At the end of the chageList() function the scoreList and nameList arrays updated by being assigned the values from these temporary arrays.

There are two other variables that play a crucial role in this function: locCounter and locCounter reverse. These two variables are used to determine the new location of each element on the top score list. These variables are incremented each time that a score is found which is lower than the current one being processed. The locCounter variable is used to hold the location of valid scores (any number higher than zero). This variable holds a score’s position from the first location in an array. The locCounterReverse variable is used to hold the location of invalid scores (0’s and -1’s). This variable holds a score’s position from the last location in the array.

Now let’s take a look at the algorithm. An outer loop cycles through 11 times to go through each score in the highscore array. A secondary embedded in the outer loop is used to compare each score with the others on the highscore list. The sketch determines the pre-liminary new location of each highscore list item by counting the number of scores that are greater than the current score.

Once the preliminary location is determined, the next step is to check if the current player’s score is repeated, and if so to adjust its position accordingly. To determine if a player has a repeated score we check the position “locCounter” in the newLocation array. If the position is not available (it is not equal to “-1”) then we loop through subsequent positions, using a while loop, until we find one that is available. This available position is then set as the new position for this list element in the newLocation array.

Once each position in the newLocation array has been assigned, a loop is used to input data into the tempName and tempScore arrays. Then these arrays are used to reset the scoreList and nameList arrays with the updated top score list.

Here is the code for the PHPconnect class that I created:

class PhpConnect {

  // variables that hold the top 10 list of scores. Each is 11 values long to help resorting when a new high score is added
  String [] completeList = new String [11];          // holds the name and score that are read from the file online
  String [] nameList = new String [11];              // holds the name of the players on top 10 list, the 11th spot holds the current player's player
  float [] scoreList = new float [11];               // holds the score of the players on top 10 list, the 11th spot holds the current player's score

  // base list URL
  String baseURL;                                    // holds the base URL that will be used to access the high score file online

  // data associated to current players' performance
  float playerScore;                              // score of current player
  float highScorePos;                             // holds the position of the player in top 10 list, or -1 if player did not reach high score list
  boolean highScore;                              // set to true if player sets a high score
  boolean setCaptureName;                         // set to true when Processing should capture a name from user
  String playerName;                              // holds the current players name (if they have a high score)
  int playerIndex;                                // holds the current letters location within the playerName string
  int nameLength = 6;                                 // holds the length of the name
  boolean ready2save;                             // set to true when Processing sends the new top score list to php application

  int [] newLocation = new int [11];              // holds the new location of each element on the top 10 list


  PhpConnect(String tURL) {
    baseURL = tURL;                               // set base URL to the parameter passed into object
    playerScore = 0;                              // score of current player
    highScorePos = -1;                            // holds the position of the player in top 10 list, or -1 if player did not reach high score list
    highScore = false;                            // set to true if player sets a high score
    setCaptureName = false;                       // holds when Processing should capture a name from user
    playerName = " ";                             // set player name to a space
    playerIndex = 0;                              // initialize the playerIndex variable
    ready2save = false;                           // holds when processing should send the new top score list to php application
  }


void loadList() {
  // create the URL used to make a high score list request
  String loadURL = baseURL + "?type=load";
  // load the top 10 list into the webPageArray (a name and score will be added to each position on the array)
  String [] webPageArray = loadStrings(loadURL);

  // loop each element in the array and breaks it down into separate name and score arrays
  for (int loadLoop = 0; loadLoop < completeList.length-1; loadLoop++) {
    // capture the top 10 elements of the webPageArray variable (in case any garbage is included at the end of the data received)
    completeList[loadLoop] = webPageArray[loadLoop];
    // split each element in the array into a name and score 
    String [] tListRecord = split(completeList[loadLoop], ' '); 

    // check if the tListRecord array has more than one element before initializing the nameList and scoreList arrays
    if (tListRecord.length > 1) {
      nameList[loadLoop] = tListRecord[0];             // save name to nameList array
      scoreList[loadLoop] = int(tListRecord[1]);       // save score to scoreList array
    }

    // DEBUG Statements
    print("name: ");
    print(nameList[loadLoop]);
    print(" - score: ");
    print(scoreList[loadLoop]);
    println();

  }   // close the for loop
}     // close the function loadList()



// to move the ARRAY count the number of figures that are larger or smaller than the current one
// checks if the current player's scores is in the top 10 and set highScore and setCaptureName to true
void checkList(float tPlayerScore){
  playerScore = tPlayerScore;                  // set player score variable
  setCaptureName = false;                      // initialize setCaptureName variable to false

  // loop through each element in the high score list to compare against current player's score
  for (int checkLoop = completeList.length-1; checkLoop >= 0; checkLoop--) {
    // tell application to capture the player's name if their score is better than the score of the player in the current array position, or if the current position is not set
    if ((playerScore < scoreList[checkLoop] && playerScore > 0) || scoreList[checkLoop] <= 0) {
      // set variable to tell application to capture name
      setCaptureName = true;
    } 
  }  
}     // close the checkList function



// function that captures the name of the user. The names for the list can be up to 5 characters long
// this function needs to be called from the keyPressed() function in the main sketch
void captureName() {
  // if current player has a high score as set by function checkList() then listen to keys
  if(setCaptureName) {
    // if delete or backspace are pressed
    if((key==char(8) || key==char(127))) {
      // and the location on the string is greater than 0
      if (playerIndex > 0) {
        playerName = playerName.substring(0, (playerName.length()-1));        // cut the last character from the name
        playerIndex--;                                                        // set the current location of the playerIndex
      } else {
       playerIndex = 0;                                                       // don't let current location get below 0
      } 
    // if return or enter keys are pressed
    } else if (key==char(10) || key==char(13)) {
      // don't do anything if the playerName variable does not contain information
      if (playerName.equals(" ") || playerName.equals("")){}
      // if the variable contains information then set highScore variable to true to notify application to sort highScore list
      else {
        setCaptureName = false;
        highScore = true;          // set high score to true
        playerIndex = 0;           // re-initialize the location on string to 0
      }
    // if a letter key is pressed then add it to the string
    } else if (key > char(48) && key < char(123)) {
      if (playerIndex == 0) {                          // if playerIndex equals 0 then initiate string
        playerName = str(key);
        playerIndex++;
      } else if (playerIndex < nameLength) {                    // only accept input if character location is lower than 5
        playerIndex++;
        playerName = playerName + str(key);
      }
    }    

  print("Capture Name - index number: ");
  print(playerIndex);
  print(" player name: ");
  print(playerName);
  println();
  }
  
}  


// function that re-sorts the top 10 list to include the new player's score
// this function should be called continuously in the draw loop
void changeList() {
  // loop through all of the elements on the list and change the values based on the change rate
  if (highScore) {
      float [] tempScore = new float [11];                                    // temporarily holds re-sorted score list 
      String [] tempName = new String [11];                                   // temporarily holds re-sorted name list
      int [] newLocation = { -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1};     // holds the location in the current array of each element on the list
      int locCounter = 0;                                                     // temporarily holds a scores new location on the array
      int locCounterReverse = 10;
      scoreList[10] = playerScore;                                            // add current player's score as the last element on the list
      nameList[10] = playerName;                                              // add current player's name as the last element on the list

      // go through each item in the array to determine its new location
      for(int change = 0; change < scoreList.length; change++) {
          locCounter = 0;                                                     // initialize the locCounter variable
          locCounterReverse = 10;
          boolean locFound = false;                                           // declare and initialized the locFound variable
          
          // compare this item in the array with all other numbers in the array
          for(int subChange = 0; subChange < scoreList.length; subChange++) {
            // if this item in the array is greater than another then add one to locCounter variable
            if (scoreList[change] > scoreList[subChange] && scoreList[subChange] > 0) {
                  locCounter++;
            }
          }
        
        // loop to adjust the locCounter variable when dealing with scores that are repeated on the list
        // loop through following steps until the location is found
          if (scoreList[change] > 0) {
              while (locFound == false) {
                    // if the location at array index [locCounter] has not been assigned yet then set locFound to true
                    if (newLocation[locCounter] == -1) { 
                       locFound = true;
                    // else if the locCounter variable has exceeded the size of the array set locFound to true
                    } else if (locCounter > scoreList.length) {
                       locFound = true;
                    // otherwise add one to the locCounter variable
                    } else {
                       locCounter++;            
                    }
              }
              newLocation[locCounter] = change;

          } else {
              while (locFound == false) {
                  if (locCounterReverse <= 0) {
                       locFound = true;
                    // else if the locCounter variable has exceeded the size of the array set locFound to true
                    } else if (newLocation[locCounterReverse] == -1) { 
                       locFound = true;
                    // otherwise add one to the locCounter variable
                    } else {
                       locCounterReverse--;            
                    }                                
                }
                newLocation[locCounterReverse] = change;
           }
       
        // set this spot on the list to the entry at the "change" location on the current array 
      }

      // loop to set the new values into the array
      for (int update = 0; update < scoreList.length; update++) {
        tempScore[update] = scoreList[newLocation[update]];
        tempName[update] = nameList[newLocation[update]];
    }

    // set the scoreList and nameList array using the temporary arrays    
    scoreList = tempScore;
    nameList = tempName;
    // tell the application that it can save the information to the server
    ready2save = true;    
    // re-initialize the highScore variable to false
    highScore = false;
  }
}


// function that saves the new top 10 list to the server
// it should be called contiously in the draw loop
void saveList() {
 
  if (ready2save) {
    String saveURL = baseURL + "?type=save";

    // loop through all of the 10 elements of the list and add them to the URL
    for (int saveLoop = 0; saveLoop < 10; saveLoop++) {
      saveURL = saveURL + "&name_" + saveLoop + "=" + nameList[saveLoop] + "&score_" + saveLoop + "=" + str(scoreList[saveLoop]);
    }  
    // call URL in order to upload new data to php application
    String [] loadStringArray = loadStrings(saveURL);
  
    // DEBUG: debugging code to view status of application
    print("saveURL: ");
    print(saveURL);
    println();

    // reset all variables to their initial state
    playerName = " ";
    playerScore = 0;
    ready2save = false;
    highScorePos = -1;
  }
}

// function that initializes the scores on the server.
void initHighScores() {
   String initializeURL = baseURL + "?type=save";
   for (int saveLoop = 0; saveLoop < 10; saveLoop++) {
       initializeURL = initializeURL + "&name_" + saveLoop + "=" + " " + "&score_" + saveLoop + "=" + "-1";
    }  
    // call URL in order to upload new data to php application
    String [] loadStringArray = loadStrings(initializeURL);
}

}

Fangs Resurrected - ICM Final Project Idea

After much brainstorming and indecision I have finally settled on my final project for the Intro to Computational Media final. My project will be to create a tricked out (or pimped out) version of the fangs invaders game. Below you will find the fangs game in its current state.

Since I last shared it in class, and on this blog, I have made a few updates to this sketch. Namely, I have added functionality to calculate and display a score, which is shown on a new page at the end of the game. This is a very small improvement in comparison to the changes I will integrate over the next two weeks.

For this project there are a bunch of updates that I plan to make. Since I may not be able to implement all of the changes I have created a prioritized list to guide my work over the past several weeks:

• Improve game scoring logic and add score to game play environment.
• Create an online high-score database (or file) using PHP scripting.
• Improve animation by adding wing movement, and making it more natural.
• Develop group behavior for the bats so that they interact and flock occasionally.
• Update look and feel of the game by adding back images and improving design.
• Add multiple stages with increasing levels of difficulty.

[picture taken from flickr user furryscalyman - Creative Commons License ]

ICM Class Notes - Using PHP - November 5, 2009

In today’s ICM class we reviewed basic concepts associated to PHP, and discussed when to use PHP (as a stand alone solution, or in conjunction with Processing).

PHP – Hypertext Pre-Processor Language

PHP was designed for server-side scripting – for development of applications that run on servers. In contrast, Processing (and Java on which it is built) was developed for client side applications – for development of programs that run on client-side computers. Originally designed to serve dynamic web content. In short, PHP enables us to store and capture information from a server more effectively and efficiently than Processing.

Here is an example of an application where PHP is used to store data for a Processing sketch. This code was written by Dan Shiffman, my professor at ITP, and it features a shared white board, where users can add new coordinates that can be viewed by all other users (anyone can also clear the board). PHP can also be used to develop a multi-user high score feature for a game developed in Processing.

How to Run PHP?

In order to run PHP you need a server or computer that is properly set-up. Most web servers support PHP, though the ones that are based on Microsoft solutions tend not to. That said, even servers that support PHP need to be properly set-up. Luckily for us, the ITP server is already set up for PHP. Many desktop and laptop computers can also be set-up to run PHP scripts.

PHP code is usually embedded in HTML documents. To embed PHP code into an HTML document the following identifiers are used: “” ends a code block. It is important to note that PHP code will not be present on the HTML source available via web browers. The reason being, the PHP code on the server is used to dynamically generate the HTML code that is sent to your computer, and viewable as source.

The Basic Elements of Coding – PHP Style

The basic concepts associated to PHP coding are similar to those used in Processing. PHP is an object-oriented language that features all of the same attributes: variables, conditionals, loops, functions, classes, etc. Here is a quick overview of some basic similarities and differences between these two tools.

Primitive Variables

All variables in PHP need to be defined/initiated by being assigned a value. To identify a variable the variable name needs to be preceded by the “$” character. However, unlike Processing, PHP does not require or support the typing of variables. This is a more flexible approach than Processing but it also opens the doors to more mistakes not being caught by the compiler.

Example variable definition code: “$ x = 5;”

Arrays

To make an array in PHP you just assign an “array(arrayElements)” to any variable name. As with primitive variables, there is no need to define the array type. To add elements to an array is simple simpler than in Processing, as shown below. It is important to note a PHP array can also hold different types of data.

• Creating an array: “$arraySample = array(0,1,2,3);”
• Adding an element to an array: “$arraySample[] = 5;”

PHP also supports associative arrays as well. These arrays indentify each data element by a name as opposed to an index number. These arrays can be used in interesting ways. They are available in Processing through Java.

• Using an associative array: “$arraySample[“fred”] = 50;”

Conditionals and Loops

The syntax for loops and conditional statements is identical to that found in Processing.

Functions and OOP in PHP

In PHP to define a function it name needs to be preceded by the identifier “function”. Similar to variables definitions, functions are not typed. Class definitions feature the same overall structure, though the constructor name is identified as “function _constructor()”. When working with instances of objects the “->” in PHP replaces the “.” in Processing.

Query Strings

Query string refers to HTML urls that reference PHP scripts and contain information that can be processed by the PHP script to dynamically generate content (HTML or other text-based documents). Since PHP primarily runs on web servers, this is the main way in which information is passed to PHP scripts. Here is an example where the identifier “name” contains data element “Julio”, “nationality” contains “brazil”, and “residence” contains “nyc”.

“http://……sample.php?name=Julio&nationality=brazil&residence=nyc”

Online, these query strings are most often used to transmit data captured from users via a web forms. Processing applications can generate query scripts that both request data and initiate other activities on the server side.

Reading Query Strings

To read values from query strings an associative array is used. PHP creates an associative array that pairs each identifier from the query string (which is always consistent) with a data element (which is variable). This array is called $_GET. Here is a sample line of code to read my name from the query string above into the variable $name:

“$name =$_GET[“name”];”

PHP Resources

• w3schools PHP tutorials
• PHP organization website
• Dan Shiffman's PHP tutorial

Miscellaneous

• Dan shared with us Coda, a development environment that enables programmers to access and edit PHP and HTML files on the server. Coda is a text editor and ftp application combined in one.
• In today’s class we got to see Josh K’s project, a sketch that generates a line that evolves in a 3D space using the principles of Perlin noise. It is a great looking visual, unfortunately, he has not yet posted the sketch online.

ICM Class Notes - Using Images and Video - October 22, 2009

Here are my notes from last week’s ICM class, where we learned how to use pictures and video in Processing. It’s taken me some time for me to get around to posting these notes because I have been focused on developing the media controller for physical computing, and working on my mid-term project. Without further ado here are my notes.

Using Images in Processing
There are two main types of activities related to using and manipulating pictures in Processing:
1. Loading and displaying images
2. Reading and manipulating the pixels

Loading and Displaying Images
In processing there is a class that handles images that is called PImage - in many ways this class is similar to the PFont object. For example, to create a new instance of an image object the loadImage(URL) function is used rather than a “new” object declaration. Here is sample code:

PFont cat;
cat = loadImage(“cat.jpg”);

The image() function is used to draw onto the screen images that are loaded. This method requires from 3 to 5 arguments, here is the complete set: image(PImageVar, xpos, ypos, width [optional], height [optional]). We can assign an image to the background() function as well, in which case the image would be displayed as the background. Here are some useful methods associated to displaying images:

• The imageMode() function allows setting of image alignment. Setting options include CENTER, CORNER (standard setting) and CORNERS.
• The tint() function enables changing the color of the image. It does not color the picture but rather removes the other colors. So if you change the tint to red, processing will set all G and B value to “0”. Tint also supports setting the transparency of an image.
• The PImage.get(xpos, ypos) function returns the color from the pixel at the coordinate xpos ypos. Get() can be used without an image to get the color of the pixel at coordinates xpos and ypos of the processing screen.
• The PImage.set(xpos, ypos, color) function sets the pixel at location xpos ypos to the color that is passed as the third argument.
• The red(color) (green() and blue()) functions determines how of that color (red, green or blue) is included in that pixels.

Loading and Displaying Videos
Processing handles videos in the same way that it handles images (it actually views videos as a bunch of standalone images. Using live video in processing is done through the Capture class. To use this class you need to add the video library to your sketch. Note that you need to use a different class to include movies, which are pre-recorded videos. Unfortunately, Processing does a bad job at handling movies though it works well with live video feeds.

To declare a Capture object you need to use the standard “new” object declaration: “myVideo = new Capture(this, xpos, ypos, refresh rate)”. When you initiate a video object processing will default to using the default system camera. The name of an alternate video source can be added between the ypos and refresh rate variables.

Before you display an image you need to read the image using the myVideo.read() function. The same image() that is used for PImage objects will display images from the camera onto the screen. All of the image processing functions discussed above can also be used to analyze video input.

Media Controller Project (and ICM Mid Term) - Phase 3

During that last several days I have been working on setting up a Processing sketch that can work with my Physical Computing media controller and serve as my mid-term project for the Introduction to Computational Media course. Long before arriving at ITP I have been interested in the design and development of media controllers. This project provided the opportunity for me to start some hands-on explorations.

In my previous post I already discussed the process for choosing the solution for playing and controlling our audio – we have decided to use Processing (and the Arduino) to control Ableton Live. Today I will provide an overview of how I developed the code for this application and some of the interface considerations associated to designing a software that could work across physical and screen-based interfaces.

My longer term objective is to create MIDI controllers using for audio and video applications using touchscreen and gestural interfaces. The interfaces that I am designing would ideally be evolved to work on multi-touch surfaces. In regards to my interest in gestural interaction, this I hope to explore through my current physical computing project and future projects.

Developing the Sketches
Since the physical computing project requires three basic types of controls that are the foundation of the media interface for my computational media mid-term, I decide to start with a focus on writing the code for these three basic elements. I set out to create code that could be easily re-used so that I could add additional elements with little effort. Here is a link to the sketch on openprocessing.org, where you can also view the full code for the controller pictured below (v1.0).

The process I used to create these sketches included the following steps: (1) creating the functionality associated to each element, separately; (2) creating a class for each element; (3) integrating objects of each class in Processing; (4) testing Processing with OSCulator and Ableton; (5) creating the Serial protocol to communicate the Arduino; (6) testing the sensors; (7) writing the final code for the Arduino; (8) testing Serial connection to Arduino; (9) calibration of the physical computing interface (whenever and wherever we set it up).

I have already made two posts on this subject (go to phase 1 post, go to phase 2 post), however, today I can attest that I have completed the vast majority of the work. The last processing sketch that I shared featured a mostly completed Matrix object that included functions for OSC communication. The serial communication protocol had also been defined.

The many additions to the sketch include creation of button and slider elements (each in its own class), a control panel (that holds the buttons and sliders), and a version of the application that features multiple button and sliders. The main updates to existing features include changes to Serial communication protocol to support additional sliders and matrices), and OSC communication code updates to ensure that messages are only sent when values change rather than continuously.

For the slider object I used the mouseDrag() function for the very first time. I had to debug my code for a while to get the visual slider to work properly. The button was easy to code from a visual perspective. The challenge I faced was in structuring the OSC messages so that I was able to send two separate and opposing messages for each click. The reason why this is important is that Ableton Live uses a separate buttons for starting and stopping clips. So I had to find a way to enable a single button to perform both functions.

The serial communication protocol update was easy to implement, so I will not delve into it here. To change the OSC communication protocol required a bit more work. I created a previous state variable in each object class to be enable verification of whether a change had occurred. The logic was implemented an “if” statement in the OSC message function.

Evolving the Controller
Here is an overview of my plans associated to this project: I plan to expand the current media controller with a few effect grids and the ability to select individual channels to apply effects. In order to do this I have to create new functions for the matrix class that enables me to set the X and Y matrix map values. I also want to work on improving the overall esthetics of the interface (while keeping its minimal feel).

From a sketch-architecture perspective I am considering creating a parent class for all buttons, grids and sliders. It would feature attributes and functionality that is common amongst all elements. Common attributes include location, size and color; common functionality requirements include detection of mouse location relative to object, OSC communication.

Questions for Class
Here is a question that came up during my development of this sketch (Dan, I need your help here). Can I use the translate, pop and pushMatrix commands to just to capture the current mouse location? This would be an easier solution to checking whether the mouse was hovering over an object.

Media Controller Project - Phase 2

This weekend I spent a lot of time working on solving the issue of how to control and manage the music clips that we want to use in our project. Our requirements are pretty straight forward, which is not to say easy to address.

Requirements for Audio Controls
We need a solution that can handle playback of looped samples and dynamic control of at least two effects to be applied on the sample (such as tempo and pitch). Ideally we would like the solution to be scalable (so we can add multiple sounds) and be able to support quantization and other techniques to ensure that the resulting sound is of good quality.

Since we are a creating a prototype to run off of a single computer we do not need this solution to be easily portable (e.g. it does not need to be easy to run on different computers).

Initial Assumptions
Due to the expertise of the team members we are using a combination of Arduino and Processing to do the heavy lifting in the areas of input sensing and data handling. After researching all of the options available in processing to manage sound, we have decided to use Ableton Live instead. Processing’s role will be relegated to interpreting the data from the Arduino to control Ableton Live via OSC.

Below I provide a more in-depth overview of my research and the solution that I have chosen. I have also posted an updated version of my sketch along with a link to the file I have created in Ableton Live for this application. Please note that you will need to set-up OSCulator in order for the sketch to work properly.

Latest Version of the Sketch
Note that I am only sharing the code for the sketch because no updates were made to the look, feel and interaction of the applet. All updates are related to enabling the sketch to communicate with Ableton via OSC.

/* IPC Media Controller Project, October, 2009
 * VIRTUAL MATRIX SKETCH
 *
 * This sketch is the first draft of the processing portion of our media controller project 
 * in its current state, this sketch only focuses on reading input from serial ports, 
 * processing this input to determine location on the virtual matrix, then provide these 
 * coordinates to other objects (such as the music generation object that we will create in the future)
 *
 */

import processing.serial.*;
Serial arduino;


import oscP5.*;
import netP5.*;
OscP5 oscComm;
NetAddress myRemoteLoc;

boolean isStarted = false;   
// Matrix-Related Variables
Matrix matrix;
final int x = 0; final int y = 1;                                      // variables to use with matrix size array
int [] cellSize = {50, 50};
int [] screenPad = {25,25};                                       // define padding between grid and screen border
int [] screenSize = new int [2];                                  // define screen size, note that we only add volSize to width, since volume knob will be place to right of screen

// Volume-Control Related Variables
int [] volSize = {0,0}; 
 
 
void setup() {
   // initialize the matrix object
   matrix = new Matrix(screenPad[x], screenPad[y], cellSize[x], cellSize[y]);   

   // instantiate the serial variable
   arduino = new Serial(this, Serial.list()[0], 9600);
   arduino.bufferUntil('.');
   
  // set frame rate
  frameRate(25);
  // start osc communication, listening for incoming messages at port 12000
  oscComm = new OscP5(this,12000);
  // set destination of our OSC messages (set to port 8000, which is the OSCulator port)
  myRemoteLoc = new NetAddress("10.0.1.3",8000);
 
   // set screen size related variables 
   screenSize[y] = int(matrix.getMatrixWidth() + (screenPad[y] * 2));
   screenSize[x] = int(matrix.getMatrixHeight() + volSize[x] + (screenPad[Y]*2));
   size(screenSize[x], screenSize[y]);                             // draw the window 100 pixels wider and 50 pixels taller
}


void draw() {
  matrix.isCellActiveMouse();
  matrix.isCellActiveSerial();
  matrix.drawMatrix();
  matrix.sendOscMessage(oscComm, myRemoteLoc);
}
 

void serialEvent(Serial arduino) {
   matrix.readSerialInput(arduino);  
}

void oscEvent(OscMessage theOscMessage) {
  /* print the address pattern and the typetag of the received OscMessage */
  print("### received an osc message.");
  print(" addrpattern: "+theOscMessage.addrPattern());
  println(" typetag: "+theOscMessage.typetag());
}


/* CLASS MATRIX
 *
 * this class holds a virtual matrix that will mimic the real world matrix.
 * It contains functions that read input from a serial port or mouse, then use that
 * input to determine the location of the object or mouse on the grid  
 *
 */

class Matrix {

  // general variables used accross class
  final int x = 0; final int y = 1;                                      // variables to use with matrix size array
  final int mouseControl = 0; final int serialControl = 1;
  
  // matrix and cell related variables
  final int [] cellNumber = {5, 5};                                      // number of cells on the horizontal axis of the matrix
  final float [] cellSize = new float [2];                               // width and height of each cell of the matrix       
  int [] matrixLoc = new int [2];                                        // location of the overall matrix
  final float [] matrixSize = new float [2];                             // the total height and width of the matrix
  float [] xCellLoc = new float [cellNumber[x]];                         // location of each cell on the grid
  float [] yCellLoc = new float [cellNumber[y]];                         // location of each cell on the grid
  Boolean [][] cellState = new Boolean [cellNumber[x]][cellNumber[y]];   // holds whether the mouse or serial object is hovering over a cell
  color activeColor = color (255,0,0);                                   // holds color of active cells
  color inactiveColor = color (255);                                     // holds color of inactive cells
  int [] previousState = {0,0};                                          // holds prevous state of the cell

  // variables for reading serial input
  int mainControl = mouseControl;
  float [] serialLoc = {0,0};                                            // holds Y reading from the serial port

  // Matrix Object Constructor
  Matrix (int XLoc, int YLoc, int cellWidth, int cellHeight) {
      matrixLoc[x] = XLoc;                                                    // set X and Y location of the virtual matrix
      matrixLoc[y] = YLoc;
      cellSize[x] = cellWidth;                                                // set the size of each cell on the grid of the virtual matrix
      cellSize[y] = cellHeight; 
      matrixSize[x] = cellNumber[x] * cellSize[x];                            // calculate width of the matrix
      matrixSize[y] = cellNumber[y] * cellSize[y];                            // calculate height of the matrix
      
      // sets the location of each cell on the grid
      for (int xCounter = 0; xCounter < xCellLoc.length; xCounter++) {
            xCellLoc[xCounter] = xCounter * cellSize[x]; }
      for (int yCounter = 0; yCounter < yCellLoc.length; yCounter++){
            yCellLoc[yCounter] = yCounter * cellSize[y]; }  
    
      // sets the status of each cell to false
      for (int xCounter = 0; xCounter < cellState.length; xCounter++) {
          for (int yCounter = 0; yCounter < cellState[xCounter].length; yCounter++) {
            cellState[xCounter][yCounter] = false; }
      }
  }  // close the constructor



 // function that returns the height of the matrix
 float getMatrixHeight(){
   return matrixSize[x];
 }


 // function that returns the width of the matrix
 float getMatrixWidth(){
   return matrixSize[y];
 }


 // function that draws the matrix on the screen
 void drawMatrix() { 
   for (int xCounter = 0; xCounter < xCellLoc.length; xCounter++){                 // loop through each element in the xCellLoc array
       for (int yCounter = 0; yCounter < yCellLoc.length; yCounter++){             // loop through each element in the yCellLoc array
         if (cellState[xCounter][yCounter] == true) { fill(activeColor); }         // if the cellState is true then change the color of the cell
         else { fill(inactiveColor);}                                              // if the cellState is false then don't change the color of the cell
         rect(xCellLoc[xCounter]+matrixLoc[x], yCellLoc[yCounter]+matrixLoc[y], cellSize[x], cellSize[y]);    // draw rectangle
       }  
   }
 }  // close drawMatrix() function


 // function that reads the input from the serial port
 void readSerialInput (Serial Arduino) {
   if (!isStarted) {                                                                 // if this is the first time we are establishing a connection
    isStarted = true;                                                                // set isStarted to true
    arduino.write("n");                                                              // respond to arduino to request more data
  } else {                                                                           // if this is NOT the first time we have received data from the arduino
    String bufferString = arduino.readString();                                      // read the buffer into the bufferString variable 
    if (bufferString != null) {                                                      // if bufferString holds data then process the data
       bufferString = bufferString.substring(0, bufferString.length() - 1);             // trim the string
       String[] serialValues = splitTokens(bufferString, " ");                          // separate the two values from the string and save them in the serialValues variable
      serialLoc[x] = float(serialValues[x]);                                                  // assign value to serialLoc[x]
      serialLoc[y] = float(serialValues[y]);                                               // assign value to serialLoc[y]
    }
    arduino.write("n");                                                              // respond to arduino to request more data
  }
}  // close readSerialInput() function
  
  
// returns an array with the unfiltered x and y locations from the serial monitor (may need to filter data based on range of serial input and requirements of music objects)
int[] getSerialXY() {
  return int(serialLoc);
}
  
// TO BE CREATED  
// function for user to set whether main input is serial or mouse based
void setMainControl(int tControlType) {
  mainControl = tControlType;
}
  
// function that sends OSC messages with input values
void sendOscMessage(OscP5 tOscComm, NetAddress tMyRemoteLoc) {
  float messageX = 0;
  float messageY = 0;

  // open new OSC messages of type x and type y
  OscMessage myOscXMessage = new OscMessage("/controlGrid/x");  
  OscMessage myOscYMessage = new OscMessage("/controlGrid/y");  
  
  // determine whether readings that are sent to OSC will originate from serial device or mouse
  if (mainControl == serialControl) {  
    messageX = map(serialLoc[x], 0, width, 0, 1);
    messageY = map(serialLoc[y], 0, height, 0, 1);
  } else if (mainControl == mouseControl) {
    messageX = map(mouseX, 0, width, 0.075, 0.125);
    messageY = map(mouseY, 0, height, 0.3, 0.7);
  }    

  myOscXMessage.add("x "); /* add an int to the osc message */
  myOscYMessage.add("y "); /* add an int to the osc message */
  myOscXMessage.add(messageX); /* add an int to the osc message */
  myOscYMessage.add(messageY); /* add a float to the osc message */
  tOscComm.send(myOscXMessage, tMyRemoteLoc); 
  tOscComm.send(myOscYMessage, tMyRemoteLoc); 

  print("X: " + messageX + " ");
  print("Y: " + messageY + " ");
  println();  
}

// returns an array with the unfiltered x and y locations from the mouse-based interface (may need to filter data based on requirements of music object)
int [] getMmouseXY() {
   int [] mouseXY = {mouseX, mouseY};
   return mouseXY; 
}


 // check if a cell on virtual Matrix is active based on the mouse location
 void isCellActiveMouse () {                                              
  int XLocMouse = mouseX - matrixLoc[x];                                        // adjust variable to account for location of Matrix within window
  int YLocMouse = mouseY - matrixLoc[y];                                        // adjust variable to account for location of Matrix within window
  isCellActive(XLocMouse, YLocMouse);                                           // call the function to check if the cell is active based on current location of mouse
 }


 // check if a cell on virtual Matrix is active based on the current physical location/state of an external object
 void isCellActiveSerial () {
   int xLocSerial = int(map(serialLoc[x], 0, 1024, 0, matrixSize[x]));                // adjust variable to account for location of Matrix within window
   int yLocSerial = int(map(serialLoc[y], 0, 1024, 0, matrixSize[y]));                // adjust variable to account for location of Matrix within window
   isCellActive(xLocSerial, yLocSerial);                                        // call the function to check if the cell is active based on current location of mouse

 }


 // function that checks whether a specific cell is Active
 void isCellActive (int tXloc, int tYloc) {
  int xLoc = tXloc;                                                                  // set the location of the X coordinate where the mouse or serial object is located
  int yLoc = tYloc;                                                                  // set the location of the Y coordinate where the mouse or serial object is located 
   
  for (int xCounter = 0; xCounter < xCellLoc.length; xCounter++){                     // loop through each element in the xCellLoc array
       for (int yCounter = 0; yCounter < yCellLoc.length; yCounter++){                // loop through each element in the yCellLoc array
 
           // check out what are the mouse or serial object is intersecting
           if (  (xLoc > xCellLoc[xCounter] && xLoc < (xCellLoc[xCounter] + cellSize[x])) &&
                 (yLoc > xCellLoc[yCounter] && yLoc < (yCellLoc[yCounter] + cellSize[y]))    ) {
                        cellState[previousState[x]][previousState[y]] = false;        // set previous grid element to false
                        cellState[xCounter][yCounter] = true;                         // set current element to active status
                        previousState[x] = xCounter;                                  // set x number of previous active cell
                        previousState[y] = yCounter;                                  // set y number of previous active cell
                 }
       }  
  }
 }
}

Making Some Noise
When we started working on this project we assumed that we would be able to use one of Processing existing sound libraries to play and modulate an audio loop. However, after doing extensive research into Minim, ESS, and Sonia, I realized that none of these tools offered the feature set that we needed for this project.

The next solution that I investigated was Max/MSP. This programming language/environment is definitely capable of providing the functionality that we are looking for. However, no one on our team has the expertise to use it nor the time to learn it for this project.

using OSC to communicate with an external music application that can provide the features we are looking for. I was happy to find out that there is a simple library called oscP5 that makes it easy to communicate from a sketch using OSC. Equally important, I also found an application called OSCulator that routes and translates OSC and MIDI messages.

Having figured out how to get the sketch to communicate via OSC and MIDI we set out to find the right application. This was an easy task in large part because both Michael and I are familiar with Ableton.

I am happy to report that we already have Ableton up and running with the virtual matrix application developed in processing, though that is not to say the sketch is finished. We still need to add start and stop buttons to the interface, along with a volume control (not to mention other improvements and ideas that have not yet been considered).

In the next day or so I will share with you more updates, including details about how the physical elements of the interface are shaping up.

ICM Class Notes on Text Processing and Serial - Oct 14, 2009

Last week’s ICM class focused on parsing strings, picking up where we left off the week before, with a short overview of serial communications. Here a list of the topics that we covered:

• Processing and XML
• Additional functions for processing text
• Serial communications

XML Libraries in Processing
XML has a hierarchical structure similar to a tree. This makes XML files easier to read than other types of content. There are many different ways that you can read XML documents in processing. Here is an overview of these options:

1. Standard string parsing functions in processing, like the ones outlined below and in my post from last week.
2. Existing processing XML libraries. Examples of these include simpleML, XMLElement, proXML. The libraries enable processing to navigate the structure of an XML file by finding a data elements by navigating through its children or parent structure.
3. Application processing interfaces (APIs) from the data source. Examples of sites with APIs include Flickr, Google Maps, etc.

Using Tokens to Parse Text
This week we were introduced to the concept of tokens and splits. Tokens are small chunks of text (these are only one character long). Here is a list of functions that leverage tokens or splits.

• split(String, SplitStringIdentifier); – This function returns multiple strings. The input copy “String” is split at the instance(s) of SplitStringidentifier. The SplitStringIdentifier is removed from the final strings.
• splitTokens(s, multiple SplitStringIdentifier); – This function is the same of split, however, it can accept multiple SplitStringIdentifiers. These are input all together within quotation marks. For example using “_.” Would look for instances where either “_” or “.” appear in the string.

Comparing Text

• equals(); – This function compares the copy contained in a string to a piece of text data. For example “string.equals(“stringData”);” is equivalent to the syntax “intVar == 3”. That said, we cannot use the Boolean operator “==” to match strings. That is why the equals() function is needed. In order to compare words regardless of whether they are lower or upper case we can use the string.toLowerCase() function.
• Regular Expressions - Regular expressions are special text strings for describing search patterns. These capabilities enable more sophisticated parsing of content than is possible through the standard functionality in Processing. This is the ideal way to perform complex data parsing and cleaning. We did not cover this in class, so research will be needed on this front.

Serial Overview
There are two main approaches to creating protocols for serial communication. The first is called punctuation, and it entails adding tokens to the data being communicated to enable the receiving computer to parse the data once received. The second approach, called handshaking, entails having each computer wait to send a message until they have received data from the other connected device.

I will not review these approaches here in detail because I have covered them in my posts from the Intro to Physical Computing class. The punctuation method is described here. I will soon add a link to the post regarding the handshake method (which is currently a work in progress).

Related notes and concepts:

• CallBack refer to methods that are called by other applications when a certain event takes place. MousePressed and SerialEvent are types of callback or event functions available in processing.
• To create a new serial object it is important to always include the Serial library (as it is not a standard processing library). Then to instantiate the object, once you’ve declared it, you need to use the following syntax: “portName = new Serial(this, “serialPortNumber”, baudRate);” The “this” argumen tells the serial object that it is setting up a communication between the serial port and this specific sketch.

Converting Applications to OOP Paradigm

Etch-a-Sketch

Since the etch-a-sketch was a simple program it was quite easy to convert it into an object-oriented structure. I decided to create one class that holds all functionality of the virtual etch-a-sketch. Though this sketch was originally created to controlled by the physical computing interface (consisting of two potentiometers) I have added the functionality to be able to use it with the arrow keys.

Here is a link to the Virtual Etch-a-Sketch app.

NYT Lists

I had issues trying to create a single NYT list object that could handle both types of list that I want to use in this sketch. Therefore, the solution that I decided to pursue was to create a parent class that includes some fully defined functions – the ones I was able to abstract to work with both lists – and two function shells – meant to be defined in the child classes. Here are a few lessons from my work on converting this sketch.

Notes on Using Inheritance

Learned that if you re-declare any variables then the functions that were declared in the parent class, which use the originally declared variable, won’t be able to update the newly declared version of that variable. The reason why I want to re-declare a variable is that it is an array, and each child classes features a different number of elements that will be stored in this array.

In the parent class I created a few functions that cycle through this array, and they seem to work. The problem is that when a function that is defined in the child class attempts to access this variable then the values, which had been updated by functions from the parent classes, are re-initialized (I got 0’s).

The solution that I used for this problem was to make the size of the array in the parent object large enough to accommodate the largest possible array size requirements from the child classes (for the case of this sketch that number was 50). To read the data appropriately, the loops in the parent functions have been set to cycle until they reach a “number of list items” variable that is defined by the user when they create and declare the object.

Using Parameters Within a Function

At first I created several functions that used the input parameters as variables within the function itself.  Though this worked in several instances, I realized that on other occasions the input parameter variables became un-useable. Therefore, now I understand that it is important to create new variables that can hold the values of input parameter for later use and processing.

Here is a link to the NYT List sketch.

IPC Class Notes, Serial Communications - week 6

This week’s class covered basic principles associated to using serial communication to get the Arduino to communicate with applications on a multimedia computer. The focus of our discussion was on using the Arduino as in input device. I've already completed the lab for this week, here's my post about how it went.

Though I have some experience in this area, having created a joystick for a game that I developed a few weeks ago, I did not understand how to create Arduino applications that leverage serial communications without using an existing library (such as firmata). This was very limiting because I was having issues with firmata.

Basic Concepts Related to Serial Communication with the Arduino
We began our exploration into serial communication by reviewing several key concepts and functions. First off, it is important to always keep in mind that only one application can use a serial port at a time (there will be no sharing of serial ports).

• ASCII – ASCII is the protocol used to transmit standard display characters, such as letter, numbers, spaces, etc. The acronym stands for American Symbolic Code for Information Interchange.
• Local versus remote: Computers cannot access each other’s code directly. They can only access messages that are exchanged via ports such as the serial one.
• FIFO – Messages are sent between the computer and a microcontroller using first in and first out organization. This is helpful because it makes sure that messages are received in the order they were sent.
• BYTE – is a setting that tells serial communication functions to send the raw value of a number using a single byte. This assumes that the value of the number will be between 0 and 255 since this is the only range of values that can be communicate using a single byte [one byte is composed of 8 bits which is equivalent to 2 to the power of 8, 256.
• DEC – is a setting that tells serial communication functions to send the value coded in ascii format. In this case the number 255 would be coded by 3 different bytes, each byte holding a separate character.
• Serial.print() – function that sends information, which it accepts as a string argument, via the serial port. The standard setting transmits information as ASCII characters (in other words, it uses DEC mode).
• Serial.Println() – similar to print() function but adds two bytes to each transmission, a line feed and a carriage return. It features the same standard setting as Serial.print().

Here is a short list of additional functions that we did not cover in class but are related to serial communication.

• Serial.write() – similar to the print() function but only sends information as BYTE.
• Serial.read() – function that reads information from the buffer. It reads the first byte of incoming data. If there is not new communications then it will return a -1.
• Serial.available() – function that reads the buffer to determine how many bytes are available for reading. The serial buffer can hold up to 128 bytes.
• Serial.flush() – function that clears the buffer. It essentially erases all data from the buffer.

Levels of agreement for serial communications

• Physical (which pins to use? how are messages physically transmitted?)
• Electrical (what level of voltage does each device require?)
• Logical (what does a pulse mean – is it considered a 1 or a 0? )
• The data (what do the 1’s and 0’s represent? How should they be grouped)
• The application (what does the groups of 0’s and 1’s signify to the application?)

Using the serial library – in Processing
On the Arduino the serial library and class is a standard part of the codeset. Therefore, you don’t need to include any additional libraries in order to send and receive data via the serial port. Unfortunately, that is not the case for Processing. That is not to say that it is hard to use the serial port in processing. Here is what you have to do:

• At the top of your sketch you need to include the serial library, since this class is not part of the core Arduino codeset.
• Declare an object from the serial class.
• Create the serial object, using three arguments: (1) “this”, which tells the computer that this program is using the serial port; (2) serial port location; and (3) connection speed.

Reading serial data with processing

• Option 1- Create a function titled serialEvent. This function is similar to mousePressed, except that it is called whenever the serial buffer fills up. This is where most of the action will happen. Be sure to pass on as an argument the serial port where the data will be available (e.g. serialEvent(Serial serialObjectPort) {code here}).
• Option 2 – use an if statement in the draw section of the program that checks if there is information available in the serial buffer (e.g. serialObjectPort.available()) {code to execute}). This approach usually works a bit slower because the draw function may be in the middle of a loop, and it will need to wait until the next loop iteration to run the code. Whereas the serialEvent() function will run regardless of the state of the draw function.

Random Additional Notes:
Cornflake: great tool to use when developing applications that leverage serial communications. Allows developer to view the content of the serial port using both byte values and ascii characters. Also makes it easy to shut off connection to port, which is important because only one application can use the serial port at once.
Check out the video and image library on processing to be able to create interfaces similar to the drawing-music one shown here.

Duplicate Nested Errors in Processing

Earlier today I came across a "Duplicate Nested" error in Processing. I spent a frustrating 15 minutes going through my code, trying to figure out what was wrong. When I did some research on the internet I found out that this error essentially meant that I had "defined a class inside a sketch of same name as [my] class" - thanks PhiLho for the tip.

This was true, I had saved the second tab on the IDE as the name of an object I created. To solve the problem all I changed the name. At first it didn't work, that is until I restarted the application. So keep an eye out for this error because if you never heard of it before you will probably be scratching your head when it happens, like I was...

Serial Input Lab and the Etch-a-Sketch Sketch

This week our lab for Introduction to Physical Computing focuses on using serial communications to enable the Arduino microcontroller to interface with other processors such as a multimedia computer. That said, this same process can be used to enable the Arduino to interface with a multitude of other microprocessors and microcontrollers.

To keep things simple, the lab exercise only requires that we set up a single analog input on the Arduino board – a potentiometer. This component is used to control a graph that is generated in processing and displayed on the monitor of the multimedia computer. I have decided to take this one step further, and to attempt to make a virtual etch-a-sketch.

Here is a link to the lab page where you can find the overview and code samples for the original lab. Another helpful link is this overview about Serial Communications from Tom Igoe’s blog. In regards to the etch-a-sketch, all of the Arduino code is included below, the processing code can be found here. But first, check out the quick video that I put together about this small project.

Managing the Communication
When communicating more than one piece of data it is important to create a syntax that can enable the receiving computer to parse out different commands.

I decide to use the following protocol for my messages to enable me to parse out the two readings from the potentiometers: The data from the first potentiometer would always be preceded by a period, while the data from the second would always be preceded by a space and followed by a period, which marked the beginning of the first data element. For example “.255 0.255 1.254 2.” And you get the point.

From the Arduino side, to make sure that I could read the serial input clearly I set the Serial.print() mode to DEC to transmit the information. Then, once the information was parsed on the other end I converted the values back to numbers (floats) so that I could use these values to determine the location of the drawing on the screen.

On the processing side I encounter more challenges than with the Arduino. The biggest problem that I had to solve was choosing a strategy for accurately reading the data from the serial port. At first I tried to read the input one character at a time, as outlined in the code sample below. I am sad to say that this approach did not work well enough because there was too much noise.

The Wrong Approach to Reading the Data

void serialEvent (Serial myPort) {
    // read the port
    readString = myPort.readChar(); 

    // if a period is found then set read the next characters as xPos values
    if (readString == char(46)) {
      readNow = "xPos"; 
      xPosString = "";

    // else if a space is found then set read the next characters as yPos values
    } else if (readString == char(32)) {
      readNow = "yPos"; 
      yPosString = "";

   // else if values have been set to be read as xPos
    } else if(readNow == "xPos") {
      if (xPosString == null) {xPosString = readString;} 
      else { xPosString = xPosString + readString;}
       xPosString = xPosString + readString;

   // else if values have been set to be read as yPos
    } else if(readNow == "yPos") {
      if (yPosString == null) { yPosString = readString;} 
      else {yPosString = yPosString + readString;}
       yPosString = yPosString + readString;
    }
} 

I was able to get things working right by learning how to set Processing’s buffer size, which governs how many bytes are received before your application calls the serialEvent function. I set the buffer size to 20 and decided to capture one full string of 20 characters at a time. Then I proceeded to read just one set of readings from each of these buffer strings. This greatly stabilized the user experience.

Unfortunately, since potentiometers have a narrow rotation range it is not possible to create the true etch-a-sketch feeling with them. On screen you can use the arrow keys to draw – at ITP we will user the small controller pictured below to play around with this (at least for a couple of days until I decide to use these parts for my next little project).

Final Arduino Code

 int analogPin1 = 0;
 int analogPin2 = 1;
 int analogValue1 = 0;
 int analogValue2 = 0;

 void setup()
 {
   // start serial port at 9600 bps:
   Serial.begin(9600);
 }

 void loop()
 {
   // read analog inputs:
   analogValue1 = analogRead(analogPin1); 
   analogValue2 = analogRead(analogPin2); 

   // reduce range to appropriate range for analog output by dividing by 4 
   analogValue1 = analogValue1 / 4;
   analogValue2 = analogValue2 / 4;
   
   // print values to serial port
   Serial.print(analogValue1, DEC);
   Serial.print(" ");
   Serial.print(analogValue2, DEC);
   Serial.print(".");

   delay(5 );                 
 }

Assignment Notes – Handling and Parsing Text

This past week our assignment for ICM was to develop a simple sketch that leverages the String parsing functionality, which we reviewed during last week’s class, to capture information from sources on the internet. For my sketch I decided to create a simple viewer that displays two New York Times lists – the most searched terms and most emailed articles lists. The list is designed to follow the mouse and scroll through items when the mouse is pressed. Here is a link to functioning application that I created (make sure to look at this work using safari rather than firefox).

In the development of this sketch I limited myself to using the following string functions: indexOf() and substring(). By far the hardest part of this project was being able to parse the webpage to capture the specific bits of data that I was looking for. Below is a quick overview of the process I used to develop the code that reads and parses content from the most emailed list. This process aligns closely with the one I outlined in my class notes earlier this week.

Before I delve into the details, I want to state the importance of choosing a good source for your data. I was lucky because the pages of the New York Times are well structured, which makes the data parsing process much easier. Here is a brief overview of the process:

1. Examining source code: once the sources had been chosen I examined the code to locate identifiers that would enable me to extract the information I was searching.
2. Developing pseudocode: once I had found the bits of information that I was looking for in the source code I created pseudocode to walk through the algorithm for reading extracting the data from the string. Note that I had already designed the display. The pseudocode is featured below.
3. Building the sketch: this is where I encountered the most issues in this process, which were related to my carelessness in transcribing the text required to identify the pieces of data that I wanted to capture.

Pseudo Code for Copy Animation

To read the title, author, and date for each item in the list I will create a loop that cycles 25 times (from 0 to 24). This loop will find the information associated to each article and save it into one of three data arrays. For each element on the list we will capture a title, date and category. Here is how we will parse the file within each loop:

First look for the div tag that marks the beginning of the content from each element in the list (class mpEntry). This tag features the item number, which correspond to the item’s ranking in the list. To use this tag dynamically insert a number into the string by converting the loop counter to a string and then adding it into tag.

The next data element is the date. The date is always located within a span tag that has a class of “date”. Once we found the index of this tag we need to add the length of the tag to identify the starting point of the date itself.

To find the end point of the date search for the span close tag. Make sure to set the nytEmailindexHolder variable to the end point that has just been located so that the search for the next piece of data begins at the right place

Next we will read the title. To find the start location of the title, first search for the start location of the link that precedes it - this link ends with “?em”>”. Then add 5 (the length of the “?em”>” string) to the index location just found above.

To find the end of the title, locate the link close tag that precedes the title. Make sure to set the search start location to the beginning to the title string we just captured.

Lastly, to capture the author information, find the div tag that has “byline” as its class – using the same strategy as above. We will then look for the closing of the “div” tags to set the end point of this string.

Learning to Control Multiple Components Through One Pin

During last week’s Intro to Physical Computing class we briefly discussed how to control multiple components through a single pin on the Arduino. Here I will cover the two approaches that we discussed – using resistor ladders, and multiplexers - and share additional information I found online.

Resistor Ladders
Resistor ladders are electrical circuits that are referred to as ladders because they are made up of repeating units or resistors that create a ladder effect on the electrical current generated. The location of each component along the ladder determines the voltage that they are able to generate. The closer a switch is to the power source the higher the voltage that will flow through the circuit when that switch is opened. The Arduino is able to determine which element on the ladders is sending the input based on this shifting level of voltage provided.

Here is a brief description of the two most common types of resistor ladders (for all I know these may be the only types). A string resistor ladder consists of a serial string of resistors. Between each resistor there is a connection to an input pin. The pin that is closest to the power source is called the most significant bit, while the farthest away is labeled the least significant bit. The R-2R resistor ladder differs from a standard string ladder in that it also features a second resistor with each switch.

Aside from enabling the Arduino to capture input from multiple switches using a single pin, resistor ladders are also often used for analog to digital conversion.


Multiplexing
Multiplexing is a second solution for expanding the number of input pins on the Arduino (demultiplexing is a solution for expanding outputs on the Arduino). One benefit of multiplexors is that they are able expand digital and analog inputs and outputs, unlike the ladder approach. Multiplexors are dedicated components that are required for multiplexing and demultiplexing.

Let’s explore the 4051 multiplexer, which is commonly used with the Arduino. Here is a link to the page regarding multiplexing on the Arduino . This component features 12 input/output pins. 4 of these pins are used to govern communications between the Arduino and the multiplexer, 3 of these pins are used to assign which pin is active, the remaining transfers the actual input or output data between the multiplexer and Arduino. The other 8 pins are the assignable pins that are used to connect other processors, sensors or actuators. Below is a schematic of the 4051 multiplexer.

Port Manipulation
While reading about the concepts described above I came across some interesting information about port manipulation on the Arduino. Over the past week I had come across several mentions of the ports on the Arduino in online examples and conversations at ITP. I am happy to report that I finally understand this capability.

The benefit of port manipulation is that it allows for lower-level and faster manipulation of the i/o pins of the microcontroller on an Arduino board. Here is a link to the page on the Arduino site that features more information about port manipulation.

The Arduino that runs an ATmega8 has the following three ports:

• B (digital pin 8 to 13)
• C (analog input pins)
• D (digital pins 0 to 7)

Each of these ports is controlled by three registers:

• DDR register: determines whether the pin is an INPUT or OUTPUT (read/write)
• PORT register: controls whether the pin is HIGH or LOW (read/write)
• PIN register: reads the state of INPUT pins set to input with pinMode() (read)

Here is an example of how these registers can be used. This example assumes that we are using port D:

• Here is a DDR register example: “DDRD = B11110000;” this sets half of the pins as outputs (the ones set to 1) and the other half as inputs (the ones set to 0).
• Here is a PORT register example: “PORTD = B11111111;” this sets all of the pins to high on a given port.

It is important to note that the “B” that precedes the binary numbers in the examples above is responsible for telling Arduino that this is a binary number. Another consideration is that the binary digits on the left refer to the higher numbered pins (opposite order of how we usually read numbers). This means that B11110000 maps to a ports pins as follows - P7=1, P6=1, P5=1, P4=1, P3=0, P2=0, P1=0, P0=0.

ICM Class Notes, Handling Strings - October 7, 2009

Topics covered during class:

• Learnings from the homework
• Displaying, reading and parsing information from the web

Learnings from the homework
While reviewing this week’s homework we came across the use of double arrays (I could not find documentation for these types of arrays on the processing website). These arrays store two dimensions of data in a matrix that can be visualized as a spreadsheet or grid (check out the fig. a below). These arrays are the basis for image and video processing since to the computer a display is nothing more than a two-dimensional array.

• Syntax for declaring double arrays: “arrayType [][] arrayName;”

Fig a. Double Array Matrix

Another interesting type of array is the ArrayList. These are a special type of array that is dynamically allocated. What this means from a practical perspective is that you can add and remove elements while the program is running. An array lists is a more efficient solution than appending normal arrays. However, if you know how many elements you will need before run time, it is most efficient to use a standard array.

This power comes at a price, to use data from these arrays you need to create temporary local variables that are often not necessary when using regular arrays. An array list also has a different set of function than a normal array (even the ones that perform the same functions have different names,).

• Syntax for declaring an array list: “ArrayList arrayName;”
• Syntax for creating the array list: “arrayName = new ArrayList();”
• Syntax for adding element to array list “arrayName.add(new Class());” - the “new Class()” in this example can be replaced by a primitive date type such as int, char, etc.

Note that an Array list does not know what it is storing. So you don’t set a type for this array when you declare it. As described by the examples above, you need only define data type when you are adding an element to the array. This means that when you need to get an element from the array you need to define what type of data you will be extracting.

• Syntax for extracting data from an array list: “Class tempVarName = (classOfArrayElement) arrayName.get(ArrayElementNumber);"

I am in no way an expert on array lists. Here is a short list of my own questions: Does the element from the array get changed when the temporary element that is linked to it gets updated? Can you add data elements of different types into an array list?

Displaying text information
During the second half of this week’s class we learned how to handle text - both how to display text and how to read text data from sources such as the Internet. I can’t wait to begin using these capabilities since data visualization is one of the areas that I hope to explore during my time at ITP. Let’s start with the basics and work our way up.

Definition of characters and strings:

• Characters (chars) is a data type for typographic characters, such as ‘a’, ‘b’, ‘c’ , and you get my drift. Chars are represented as ‘single’ quotes.
  
• Strings (String) is a data type that contains an array of chars. Unlike traditional arrays you don’t need to define the size of your string when you create it. Strings are represented by “double” quotes.
  

Adding text to your processing sketch: There are two ways to add text to a sketch in processing, loadFont() and createFont(). To use loadFont you need to convert the font into an image (vwl file) prior to compiling the program. Aside from this difference the process is quite similar. Here is a step-by-step description:

1. Declare a variable of type PFont
2. Define the variable usingloadFont() or createFont(). Do this during the sketch set-up because it requires a good amount of time.
3. Use textFont() function to set the font type and size that you want to use for a single use of the text().
4. Display text using the text() function. Method has several arguments: string, x and y coordinates, and width and length (optional).

So what are the benefits of each approach? loadFont() is a good solution when you are using a font that your users may not have installed on their own computers because the font will be embedded in the program. On the downside, since the font is converted to an image file at a specific size it can also get pixilated if it is used at a larger size than it was originally intended for.

On the other hand, createFont() enables to load a font at the moment the application is running. It also supports resizing of fonts without pixilation. However, if the person running the application does not have the selected font then the system defaults to an available font, which may look horrible with your design.

Capturing information from the Internet
For the most part, being able to take data from a website and using that data to impact a sketch takes a lot of detective work. The reason being you need to look at the source code from a website to understand how it is structured so that you can determine if there is a standard syntax that can enable your sketch to consistently find the information you want to capture. Here is an overview of the entire process

1. In practice this first step involves looking for identifiers within HTML/XML code that precede and follow the golden nugget of information you seek. It is always easier to use XML feeds than HTML code (and firefox is better to investigate XML than Safari). If using XML code you should leverage processing’s XML library. Once you have found the source on the internet from which you want to import data go to the next step.

2. Load the page into processing as a string using the loadStrings(URL) function. This method loads the document into an array of strings, each line is put into a separate element in the array. This means that when we are declaring a variable to import a text document we need to set that variable as an array – “string [] lines = loadString(sampleURL);”

3. Next use the join() function to put the contents of this array into a single string. The syntax for the join() function is “Join(string array to be joined, element to add between array elements)”. For example join(htmlPageArray, “ “) would join all the lines from a string array with a space between each element in the array.

4. Use string parsing functions described below to locate the data we want within an XML or HTML document/code. All of these methods tare part of the String class).

Parcing information from the Internet
First off you need to find the location of the data you are looking for by using the unique identifiers that you discovered during step one described above. Then you will use this information to extract data from the string. Remember that you can use a loop to look for multiple similar elements on a page. Here is the step-by-step:

1. The function stringVar.indexOf(subStringSearch) returns the location within the stringVar where substring starts. If the subStringSearch is not found then it returns a -1 value. In order to find the start location of a sub string we need to add the length of the subStringSearch variable to value returned by this function.
2. Once you have the location of the data use the stringVar.subString(start position inclusive, end position exclusive) to get the sub string that exists between the start and end point. It is important to always keep in mind how the start and end points of this function are handled differently.