A group from ITP was enjoying the bitter weather at Mount Snow, in West Dover, VT on our (now annual?) Snowbunnies trip. This crash was late in the day on a wide open trail. I accidentally disengaged my heelside edge for a moment, causing me to rotate slightly clockwise and slide laterally. Moments later, my heelside edge caught again, now on the downhill side, causing me to quickly flip backwards onto my head … thankfully I was wearing a helmet. After that I can’t recall what exactly happened, but I know that it involved a lot of tumbling which my right ankle just couldn’t weather.

The data was recorded at 10Hz and includes GPS coordinates, heading, estimated speed and altitude and the accelerometer values. I used a simple java / processing application to graph the log. In the video clip below I’ve applied low pass filtering to the raw acc data and used that to control a 3d box, representing my orientation during the crash.

The log file is included here in CSV format if anyone wants to check it out or make an awesome visualization of it! There is some weirdness at the beginning of the file…perhaps as Core Location was getting a GPS fix. [crash datalog]

I’m logging the acceleration forces at the handlebars of my bicycle while riding through New York City. The body has roughly three contact points with a bicycle, the hands at the handlebars, the “seat” at the saddle, and the feet at the pedals. The downward force of the rider’s weight and pedaling force and the upward forces of the bicycle rolling over uneven ground are distributed over these three points. I was interested to see just what kind of forces are “pushing back” that I may not be aware of, myself lost in the act of simply keeping the bicycle upright and safely navigating through traffic.

To contextualize the raw accelerometer data I also tracking GPS location and eventually geocoding the raw data in software. The bicycle sensors are being transmitted via Bluetooth to a mobile phone and the data is logged with a custom written (but now open-source!) python script. Below is the first draft of the visualization.

I intend to add more sensors to record internal forces to see if there is a physiological response (HR, GSR, breathing, pressure on contact points) to external factors such as speed, traffic / road / weather conditions, time-of-day etc.

Video demonstration of the initial visualization.

Camera vision. Started working with openFrameworks to do the visual tracking of objects on the table. Unexposed portions of developed film negatives block most visible light and let IR light pass. Using this as a filter over the built-in iSight camera, I was able to test a rudimentary camera tracking system. It would likely be better to use a more robust library like openCV, but writing the tracking myself helped me to learn about how it works.

Still have to merge these two components. It looks like I’ll have to rewrite the flocking system in C++ since Processing/Java is getting bogged down. Ameya is working on the web side of the project – messing with a linux-based router to handle the proxy and packet sniffing as well as a database system to manage the data.

The table was built throughout this past week; we still need to mount the camera and projector in an effective way for under projection and sensing…any advice?

There is something really nice about the amplification of a small facial movement and the larger audio/visual response of the sketch. It’s also nice to interact in a handsfree way. Oh! Fun. Code after the video.

Arduino code:

/* brauswitch
* robert carlsen | robertcarlsen.net
* 2-2008
*
* the brauswitch is a headband mounted switch activated by raising the eyebrows.
* the prototype version is made from burlap with conductive fabric on opposite sides
* of a small gap in the headband just above the eyebrows. raising the eyebrows closes the gap
* which closes the switch. the position of the headband needs to be adjusted carefully for proper
* action - however once situated well the brauswitch works very consistently.
*
* this code sends a byte via serial when the switch is closed:
* 1, 2 or 3 for left, right and both switches respectively
*/

#define LEFT_BROW 8
#define RIGHT_BROW 9
#define LED 3

void setup() {
Serial.begin(9600);

pinMode(LEFT_BROW, INPUT);
pinMode(RIGHT_BROW, INPUT);
pinMode(LED, OUTPUT);

}

void loop(){
int var1 = digitalRead(LEFT_BROW);
int var2 = digitalRead(RIGHT_BROW);

byte msg = 0;

if(var1 == HIGH){
digitalWrite(LED, HIGH);
msg += 1;
delay(10);
digitalWrite(LED, LOW);
}

if(var2 == HIGH){
digitalWrite(LED, HIGH);
msg += 2;
delay(100);
digitalWrite(LED, LOW);
}

if(msg>0)
Serial.print(msg,BYTE);

delay(100);
}

Processing (java) code:

// Project:         brauswitch
// File:             Brauswitch.java
// Created by:         rcarlsen, Feb 21, 2009

// Imports
import processing.core.*;
import ddf.minim.*;
import processing.serial.*;

public class Brauswitch extends PApplet {
Serial myPort;

// holder for the incoming data
byte[] data = new byte[1];

// color array
int[] c = {0x33000000,0x33ff0000,0x3300ff00,0x330000ff};
int cIndex = 0;

int timer;
int timeout = 1000;

Minim minim;
AudioSnippet bothSound,leftSound,rightSound;

public void setup() {
size(500,300);
smooth();
background(0);
noStroke();

//println(Serial.list());
myPort = new Serial(this,Serial.list()[0],9600);

minim = new Minim(this);
bothSound = minim.loadSnippet("beat.wav");
leftSound = minim.loadSnippet("msgstart.wav");
rightSound = minim.loadSnippet("msgend.wav");
}

public void draw() {
// draw a partially transparent rect over the previous frame
fill(0x33000000);
rect(0,0,width,height);

// read the serial data is available
if(myPort.available()>0){
println(myPort.available() + " bytes available");

//only expecting one byte
data = myPort.readBytes();
myPort.clear();
}

// act on the read data. it will be 0,1,2,3
if(data[0]>0){
println("Data read: " + data[0]);
cIndex = data[0];
// clear the data
data[0] = 0;

switch(cIndex){
case 1:
if(!leftSound.isPlaying())
leftSound.loop(0);
break;
case 2:
if(!rightSound.isPlaying())
rightSound.loop(0);
break;
case 3:
if(!bothSound.isPlaying())
bothSound.loop(0);
break;
}

// keep the timer going
timer = millis();
}

// fade out if the brauswitch is open
if(millis() - timer > timeout){
cIndex = 0;
if(bothSound.isLooping())
bothSound.play(); //finish the sound and stop
if(leftSound.isPlaying())
leftSound.pause();
if(rightSound.isPlaying())
rightSound.pause();
timer = millis();
}

// draw the indicator ellipse
fill(c[cIndex]);
ellipseMode(CENTER);
ellipse(this.width/2, this.height/2,200,200);
}

public void stop()
{
// always close Minim audio classes
bothSound.close();
leftSound.close();
rightSound.close();
// always stop Minim before exiting
minim.stop();

super.stop();
}
}

Some of my notes from fair:
Infrared control.
Lots of robot kits. Solar powered vs. battery powered.
Grouped into prefab and modular kits.
Sound and light sensors, too.
Example: HexPods. Overheard vendor discuss user testing: Kids want control and speed.
Slot car systems. Even here there is much licensing. (Nintendo – Mario Cart)
Figurine playsets. Thematically related. Realistically detailed. Schleich.
Glow strings and kits.
Materials: Lots of plastic (PVC), lots of wood.
Many stuffed animals.
Lots of board games and educational toys. Brain teasers.
Flying toys. Planes/helicopters.
Tents and other enclosures.
Saw remote controlled drawing robots. Reminded me of Chris Cerrito’s project
Pedal powered cart. Awesome. Disc brakes and 7 speed shifting.

So, an idea for the toy design class:
Handheld car/wheeled bot. Move it around, let go of it, then it will continually mimic the previous movement. Inspired by yellowtail, but physical. I feel that I’ve seen something similar from a research lab, but can’t find the site. (anyone want to point it out to me?)

Here’s a (rough) software prototype: TraceBug

Maybe it could be a game? Have to make an increasing number of discreet moves without hitting anything? Maybe it could be like lightcycles (from the movie Tron) – remember the path and try not to cross over it? Maybe there is an obstacle course? Maybe you are trying to trace and existing path?

Of course, you can still roll the car around even when it’s without power.

So, that’s the rough idea.

Here’s a quick sketch of the possible output. There is also an animated version with speed controls.

This is all very rough, but I wanted to mock something up before investing too much work into prototyping it.

Regardless of the data source, for the output side I’m driving the analog decibel meter with an Arduino using pulse width modulation (PWM). Inside it seems to be a small electromagnet whose field moves the needle to the right. The movement is counteracted by several springs which keep the needle trying to pull back to the left.

It seems to take minimal current < 1mA at 5V to peg the needle. The face of the meter indicates “1mW 600 Ohm” which I assume means that the max input is 1mW and the resistance of the unit is 600Ohm. (Please correct me if I’m wrong). I’ve installed a 220Ohm resistor and small LED in series to provide more load to the PWM pin which allows finer steps in the meter as well as protects the Arduino from any possible reverse current as the needle’s springs pull it back past the electromagnet. I likely don’t need that, but it wouldn’t be an ITP project without an LED (it’s not blinking, though).

In this demo the Arduino is receiving data via a serial connection from a simple Processing sketch to control the position of the needle. In the above mentioned configuration I’m mapping the input value to a PWM output range of 0-40. Sending byte 255 will change modes in the program. The default mode is to accept ASCII values of 48-52 (that’s 0-9) so I can test in a terminal. The second mode will accept all bytes bytes of 0-254, and the third mode simply reports back the received values.

Next step is to figure out exactly what to do with the meter; since it accepts serial data it is pretty modular.

I decided to ignore all the nice easy stuff and get right to speaking to the Meggy’s frame buffer, a 192 byte long array containing the values for each RGB pixel. I wrote up a Processing sketch on a MacBook Pro to capture a video image, scale it to 8×8, format the image in a Meggy framebuffer style array and send the whole thing via serial. The most challenging part was figuring out the framebuffer format, which although documented by EMSL was difficult for me to work out initially.

Also, the colors are not correct on the first try, and the RGB elements respond with different brightnesses than each other. The source code from the Meggy library provided some insight on suitable adjustment levels, which the Processing script is doing before sending along the pixel values. Fun times…enjoy the video (may help to squint at the pixels)! Code below the video.

Arduino code:

/*
  MeggyJr_VideoDisplay
 2009 Robert Carlsen // robertcarlsen.net

 Accepts serial data of a video stream from a desktop counterpart
 Expects a header byte of 128, then 192 bytes of pixel data (<= 127)
*/

#include <MeggyJr.h>
MeggyJr Meg;

// stores how many bytes have been received for the current frame:
byte index = 0;

void setup()
{
  Meg = MeggyJr();    // Required.
  Serial.begin(57600);
}  

void loop()
{
  // check if data has been sent from the computer:
  if (Serial.available()) {
    // read in the current byte:
    byte val = Serial.read();

    // look for the header:
    if(val == 128){
      // the header byte was found, the beginning of a new frame in next:
      index = 0;
      return;
    }

    // write the current info directly to the MeggyFrame buffer
    // this may cause tearing or other artifacts...please clean up if you like 
    // the frame data has been packed in the expected order by the sender program
    // also, only the low four bits are used for pixel color. "& 0x0F"  masks them.
    // i'm thinking of packing other data, such as vertical blanking
    // pixel index or "timecode" in the high bits.
    Meg.MeggyFrame[index] = val & 0x0F;
    index++;
  }

  // don't really need a delay here
  // delay(7);
}

Processing sketch:

/*
  MeggyVideoSend
  2009 Robert Carlsen // robertcarlsen.net

  Captures the video camera image, downsamples it to 8x8
  and packs the pixel data in a format compatible with the Meggy Jr RGB.
  Finally, send the pixel data to the Meggy

  Requires the complementary MeggyJr_Video firmware loaded on the Meggy
*/

import processing.video.*;
import processing.serial.*;

Serial myPort;  // Create object from Serial class
Capture myCapture; // set up a video capture object

// the LEDs respond with varied brightness.
// this adjustment will color correct them.
// feel free to experiment.
float[] adj = {0.6,1.0,0.5}; // red,green,blue

void setup() {
  size(300,300);

  // open a serial port to the Meggy. you may need to change the 0. check the list in the console.
  String portName = Serial.list()[0];
  myPort = new Serial(this, portName, 57600);

  // start a video capture:
  myCapture = new Capture(this, 8, 8, 30);

  // if you have trouble with the above, uncomment the below and replace "Camera Name" with the appropriate device.
  // println(Capture.list());
  // myCapture = new Capture(this, width, height, "Camera Name", 30);
}

void draw() {
  // create an array for the pixel data:
  // one header byte and 192 bytes for pixels (64 pixels * 3 bytes / pixel)
  byte[] c = new byte[193];

  // read the pixels from the video capture:
  myCapture.loadPixels();

  // header byte:
  c[0] = (byte)128;

  // loop through all the pixels in the video image
  // adjust the colors and pack the bytes in the correct order:
  for(int i = 0; i < 64; i++){
    // adjust colors:
    // for an explanation of what's going on, look up "bit shifting" on the Processing site.
    int r = (int)(adj[0]*((myCapture.pixels[i]>>16) & 0xFF));
    int g = (int)(adj[1]*((myCapture.pixels[i]>>8) & 0xFF));
    int b = (int)(adj[2]*((myCapture.pixels[i]) & 0xFF));

    // the Meggy expects 0-15 values only.
    // right shift ">>4" essentially divides our 8-bit values in Processing (0-255) by 16 giving a range of 0-15.
    // the Meggy packs the pixels into a long array, similar to the pixels in Processing.
    // however, each pixel is stored in 3 separate bytes rather than one big integer.
    // the (messy) conversion is below. we're also adding 1 to the index to account for the header byte.
    c[24*(i/8)+i%8+17] = (byte)(r>>4); // r: index + 16
    c[24*(i/8)+i%8+9] = (byte)(g>>4); // g: index + 8
    c[24*(i/8)+i%8+1] = (byte)(b>>4);  // b: index
  }

  // send the whole frame data out via serial:
  myPort.write(c);

  // draw the current frame to the computer screen for preview:
  image(myCapture,0,0,width,height);
}

// read in a new frame of video if available:
void captureEvent(Capture myCapture) {
  myCapture.read();
}

Applet and code

While the graphing application was certainly useful in tuning the system, it was confusing for viewers to understand. I don’t really view this as a fault of the design since it was intended to fulfill a specific need in developing the project. However, I really would like to convey the moments of traffic’s encroachment into a cyclist’s personal space.

I was directed to look into the field of proxemics and the work of Edward Hall by our “Visualizing the Five Senses” professor, Jane Nisselson (Virtual Beauty). What interested me most was the depiction of personal space into four categories: Intimate, Personal, Social and Public. Although this work primarily seems to deal with intrapersonal relationships I wondered if the categories could be used to express the space around a cyclist…specifically to communicate the space to non-cyclists.

Here are several images of the resulting visualization. The project is animated – I’ll an example of the movement online in the next few days, with the source code.