Much of this project stems from two separate influences. First, William Gibson’s “Neuromancer” where he depicts the protagonist using electrodes/glasses to jack into cyberspace, which is depicted as a three-dimensional space largely visualized as colored geometric shapes. Secondly, my general jealousy of individuals whom describe themselves as synesthetes, those who experience certain senses crossing over, ie. seeing certain letters as colored or hearing certain visual motion.

An article in the journal Cortex (#44, May 2008,  Dominic H. ffytche), describes a study which employed flashing goggles to induce visual hallucinations for the purpose of analyzing brain activity with MRI. The study indicates that flashing frequencies between 5-30Hz at a certain intensity caused participants to describe perceiving hallucinations (Purkinje patterns) and produced a measurable change in the brain activity.

I’d like to use this visual stimulation to represent a small portion of our sense of proprioception. The ideal scenario would be to enable a user to generate an association between certain Purkinje patterns and the orientation of the body relative to itself. I’m working closely with Andrew Styer on the input devices and we’ve been discussing the best sensing for our end. Trying to visualize all of proprioception in this manner would be arrogant at best, so for this prototype we are looking to limit our focus to the upper body, possibly the arms alone.

Initial user testing of the glasses have provided strong reactions and most participants report perceiving patterns which change in response to frequency shift. Participants generally indicate that the experience is pleasant, however the prototype glasses may provide too much illumination intensity, overpowering the effect.

Next steps are to experiment with various sensors / inputs and to determine the levels of discernible patterns within the available frequency range. Another direction for exploration is the event as frequencies change. Several participants have expressed strong reactions when the patterns initially change. Perhaps alternating between frequencies could provide another dimension to the expressiveness of the glasses.

Corey Menscher and I worked together on ThinkBig for the Whole Body Interface exercise in Thinking Physically. Corey had the basic idea of making soft switches which would be foot activated. Initially he was going to make three which you’d have to skip between.

We had a discussion about other implementations of the idea, including a hopscotch-type arrangement but ended up riffing off the large keyboard depicted at FAO Schwartz in the movie Big. To this end we made 8 switches which would be mapped to notes on a musical scale. After more discussion we decided to allow the switches to be arbitrarily positioned, including on walls, and the actions would trigger sound samples rather than notes.

Building the soft switches was easy enough – two layers of conductive fabric with a layer of dense foam between them. We wanted to put them on flexible surfaces, but for the prototype affixed each to a small wooden block.

The class seemed to enjoy using them, especially when they were on the wall rather than the floor. The scale of the interface encouraged multiple simultaneous users to play, and the distance between each switch introduced a natural limit to how quickly you could move from one switch to the next. As with the Brauswitch the output was simple sound samples, but it would be interesting to experiment with other actions.

Video of the class trying it out below. Enjoy.

There is something really nice about the amplification of a small facial movement and the larger audio/visual response of the sketch. It’s also nice to interact in a handsfree way. Oh! Fun. Code after the video.

Arduino code:

/* brauswitch
* robert carlsen | robertcarlsen.net
* 2-2008
*
* the brauswitch is a headband mounted switch activated by raising the eyebrows.
* the prototype version is made from burlap with conductive fabric on opposite sides
* of a small gap in the headband just above the eyebrows. raising the eyebrows closes the gap
* which closes the switch. the position of the headband needs to be adjusted carefully for proper
* action - however once situated well the brauswitch works very consistently.
*
* this code sends a byte via serial when the switch is closed:
* 1, 2 or 3 for left, right and both switches respectively
*/

#define LEFT_BROW 8
#define RIGHT_BROW 9
#define LED 3

void setup() {
Serial.begin(9600);

pinMode(LEFT_BROW, INPUT);
pinMode(RIGHT_BROW, INPUT);
pinMode(LED, OUTPUT);

}

void loop(){
int var1 = digitalRead(LEFT_BROW);
int var2 = digitalRead(RIGHT_BROW);

byte msg = 0;

if(var1 == HIGH){
digitalWrite(LED, HIGH);
msg += 1;
delay(10);
digitalWrite(LED, LOW);
}

if(var2 == HIGH){
digitalWrite(LED, HIGH);
msg += 2;
delay(100);
digitalWrite(LED, LOW);
}

if(msg>0)
Serial.print(msg,BYTE);

delay(100);
}

Processing (java) code:

// Project:         brauswitch
// File:             Brauswitch.java
// Created by:         rcarlsen, Feb 21, 2009

// Imports
import processing.core.*;
import ddf.minim.*;
import processing.serial.*;

public class Brauswitch extends PApplet {
Serial myPort;

// holder for the incoming data
byte[] data = new byte[1];

// color array
int[] c = {0x33000000,0x33ff0000,0x3300ff00,0x330000ff};
int cIndex = 0;

int timer;
int timeout = 1000;

Minim minim;
AudioSnippet bothSound,leftSound,rightSound;

public void setup() {
size(500,300);
smooth();
background(0);
noStroke();

//println(Serial.list());
myPort = new Serial(this,Serial.list()[0],9600);

minim = new Minim(this);
bothSound = minim.loadSnippet("beat.wav");
leftSound = minim.loadSnippet("msgstart.wav");
rightSound = minim.loadSnippet("msgend.wav");
}

public void draw() {
// draw a partially transparent rect over the previous frame
fill(0x33000000);
rect(0,0,width,height);

// read the serial data is available
if(myPort.available()>0){
println(myPort.available() + " bytes available");

//only expecting one byte
data = myPort.readBytes();
myPort.clear();
}

// act on the read data. it will be 0,1,2,3
if(data[0]>0){
println("Data read: " + data[0]);
cIndex = data[0];
// clear the data
data[0] = 0;

switch(cIndex){
case 1:
if(!leftSound.isPlaying())
leftSound.loop(0);
break;
case 2:
if(!rightSound.isPlaying())
rightSound.loop(0);
break;
case 3:
if(!bothSound.isPlaying())
bothSound.loop(0);
break;
}

// keep the timer going
timer = millis();
}

// fade out if the brauswitch is open
if(millis() - timer > timeout){
cIndex = 0;
if(bothSound.isLooping())
bothSound.play(); //finish the sound and stop
if(leftSound.isPlaying())
leftSound.pause();
if(rightSound.isPlaying())
rightSound.pause();
timer = millis();
}

// draw the indicator ellipse
fill(c[cIndex]);
ellipseMode(CENTER);
ellipse(this.width/2, this.height/2,200,200);
}

public void stop()
{
// always close Minim audio classes
bothSound.close();
leftSound.close();
rightSound.close();
// always stop Minim before exiting
minim.stop();

super.stop();
}
}

In terms of human interaction, physical behavior has five primary functions: (1) to express emotion; (2) to regulate interpersonal interactions; (3) to present one’s personality to others; (4) to convey interpersonal attitudes and relationships; (5) to replace or accompany speech.

I was interested in drawing from a fairly common position I’ve found myself in lately: being deep in concentration reading or brainstorming. Analyzing the common body positions I’ve observed myself and others in when in a similar state, here is my proposed gesture for “I’m busy/tired/frustrated/overwhelmed – go away / leave me alone right now.”

Three fingers touch the face. Index finger above the outside corner of the eyebrow. Middle finger on the forehead above the inside corner of the eyebrow. Thumb just below the cheekbone. Head can be, but not necessarily be downturned as if reading a book or screen.

I started with a gesture for having a headache – squeezing the temples or rubbing the forehead with the tips of the fingers of both hands. Then rubbing the forehead with fingers and thumb on opposite sides of the face. This gesture is modified from those.

Folks seemed to be unsure of the gesture at first, but were receptive to try it out. The finger positions varied slightly, but still are recognizable. The expression of the eyes also seems to play into the gesture greatly.

Each of the six moves we created were inspired by the wearable digital switches presented earlier in class. They are labeled in a some what literal way, generally after the motion or body part the switch employed.

The names are:

1. Heels
2. Brows
3. Clap
4. Chicken
5. Elbow
6. HeadLeg

The dance as determined by rolling the die followed two simple rules:

• Each move must appear once
• No move can appear more than twice

Our dance became:
Brows, Elbows, Brows, Heels, HeadLeg, Chicken, Chicken, Elbows, Heels, Clap
Try it out…it’s as much fun as it is ridiculous looking.

The Brauswtich is a digital switch to be worn on the forehead, just above the eyebrows. Raising one or both eyebrows will close the switch. There is a small gap between the upper and lower portions of the headband. It is made with a heavy burlap; the upper portion is stiffer than the lower portion which generally moves less as the eyebrows are raised and enables the switching motion.

It was designed without a specific output in mind; the task was to work within the confines of the assigned body part – in this case the head and torso. In class we attached each of our switches to an Arduino +WaveSheild and used them to trigger sound effects.

When adjusted correctly, it takes a very subtle movement to activate the switch although larger movements will work as well.

Nothing groundbreaking here, but this was my first experience working with conductive fabric to build a wearable circuit. I may be hooked.

Corey wearing the Brauswitch.

rob faludi came in to speak about affordances and provided the treasure hunt:

• an affordance for you.
• something that object doesn’t afford you
• and affordance only for someone else
• an affordance for a non-human
• a perceived affordance that isn’t real
• a real affordance that isn’t (easily) perceived
• an attempt to lock out an affordance
• an attempt to extend an affordance
• any affordance you think will surprise us [the class]

here are the results of my treasure hunt. i’ll fill in missing items as i come across them.

i’ve always been a cat person. when i lived with my first cat i’d watch his ears swivel as he would listen to the sounds around us. sometimes i’d make quiet noises when he was looking away to see if he’d turn his ears toward me.

it is easy, of course, to accomplish a shadow of his ability by cupping my hands behind my ears…but i wanted to try something much more foolish.

enter the {h}ears. cardboard with hook and loop fasteners is curled around the ear into a horn, similar to an old phonograph or megaphone. they are light enough to stay put without being held while walking. hearing is (arguably) more directional while they are worn. this first prototype evokes more of a mouse ear than a cat ear, but there is always version two.

back to the drawing board – literally. it was really nice to draft by hand for a change rather than start at the keyboard. i may be onto something

here’s the template for your own {h}ears.

My initial notes:
People seem to be aloof or wary of unfamiliar others. A shared experience seems to break down this distance. Observing a humorous or traumatic event. An emergency where people have to work together quickly. A spontaneous celebration. Whether it is a positive or a negative experience, a remarkable experience is what will break down barriers.

Apparel. A provocative piece of clothing will stimulate conversation. Affiliation though style, political or sports slogan can serve to both connect and to alienate others.
Gadgets – a sighting of an iPhone, and first generation iPod before it would generate interest enough to overcome avoidance of strangers. Someone wearing headphones is interpreted as a signal that they want to be left alone.
Gesture. Smiling might convey various messages given context…however they would still be different than a furrowed brow. Eyes – looking at someone vs. averting gaze. Culturally determined?

On another level, what about established relationships? I’ve been focusing on unfamiliar relationships between people – strangers, in a city perhaps. What about acquaintances, friends, co-workers, family, partners (an all the variant types here). What about opening up more in an established relationship? Would that be limited focused on emotion?

After a quick message passing with Kate who gave me some great advice – don’t overthink it – I came up with a simple idea: “what about a ‘Hi 5′ shirt? Something really graphically simple and aesthetic. without words, a simple gesture and the image should convey the intention.”

I’ll know if it works later today…