The project page is: Pocket OCR

The source code is available at github: http://github.com/rcarlsen/Pocket-OCR

There is certainly a lot of improvement to be made. Automatic color correction. Page layout recognition. Perspective correction…the list could go on. The code is there, so…fork away!

(the thumbnail is a bit tongue-in-cheek…but honest. good conversion requires a good source image: well-lit, macro, focused and tightly cropped seems best)

This visualization traces the routes I rode throughout the fall, highlighting big bumps. Areas of New York which caused me to experience lots of bumps begin to glow bright red.

The show is a great time; I’d highly suggest coming to see the myriad of projects this year. There’s a online guest book with project map at ITPGuestbook.

i finally got around to building a proof of concept implementation of tesseract-ocr for the iPhone. months ago, i documented the steps which helped to get the library cross-compiled for the iPhone’s ARM processor, and how to build a fat library for use with the simulator as well. several folks have helped immensely in noting how to actually run the engine in obj-c++. thanks to everyone who has commented so far.

anyway, below is a short video of the POC in action. the basic workflow is: select image from photo library or camera, crop tightly on the box of text you’d like to convert, wait while it processes, select / copy or email text.

there are loads of improvements which could be implemented (image histogram adjustment, rotation / perspective correction, automatic text box/layout detection, content detection – dates, links, contact information…) but this is a nice point to stop and document.

i realize that there are several OCR applications available for the iPhone, including a few which also run the engine on the device rather than handing it off to a web service. this started as an educational project on cross-compiling, and to fill a personal want for a handheld OCR app of my own. for these reasons, i’m going to open-source the entire app. look for it after this semester ends when i’ll have some more time to properly document the code. in the meantime, enjoy these code snippets demonstrating how to initialize the engine and process an image.

Initialize the engine:

    NSString *dataPath = [[self applicationDocumentsDirectory] stringByAppendingPathComponent:@"tessdata"];
    /*
     Set up the data in the docs dir
     want to copy the data to the documents folder if it doesn't already exist
     */
    NSFileManager *fileManager = [NSFileManager defaultManager];
    // If the expected store doesn't exist, copy the default store.
    if (![fileManager fileExistsAtPath:dataPath]) {
        // get the path to the app bundle (with the tessdata dir)
        NSString *bundlePath = [[NSBundle mainBundle] bundlePath];
        NSString *tessdataPath = [bundlePath stringByAppendingPathComponent:@"tessdata"];
        if (tessdataPath) {
            [fileManager copyItemAtPath:tessdataPath toPath:dataPath error:NULL];
        }
    }

    NSString *dataPathWithSlash = [[self applicationDocumentsDirectory] stringByAppendingString:@"/"];
    setenv("TESSDATA_PREFIX", [dataPathWithSlash UTF8String], 1);

    // init the tesseract engine.
    tess = new TessBaseAPI();

    tess->SimpleInit([dataPath cStringUsingEncoding:NSUTF8StringEncoding],  // Path to tessdata-no ending /.
                     "eng",  // ISO 639-3 string or NULL.
                     false);

Process an image. This should be threaded as it’s a heavy process:

    CGSize imageSize = [uiImage size];
    double bytes_per_line	= CGImageGetBytesPerRow([uiImage CGImage]);
    double bytes_per_pixel	= CGImageGetBitsPerPixel([uiImage CGImage]) / 8.0;

    CFDataRef data = CGDataProviderCopyData(CGImageGetDataProvider([uiImage CGImage]));
    const UInt8 *imageData = CFDataGetBytePtr(data);

    // this could take a while. maybe needs to happen asynchronously.
    char* text = tess->TesseractRect(imageData,
                                     bytes_per_pixel,
                                     bytes_per_line,
                                     0, 0,
                                     imageSize.width, imageSize.height);

    // Do something useful with the text!
    NSLog(@"Converted text: %@",[NSString stringWithCString:text encoding:NSUTF8StringEncoding]);

    delete[] text;

Enjoy the video!

Slashdot users are seeking karma. However, gaining positive karma at Slashdot is just a means to an end; Slashdot users are seeking (limited) power and status among their peers in the form of fleeting moderator access for the vibrant comments component of the highly active, technology-focused news aggregation site. Moderators are chosen from among the registered users using a somewhat obscure algorithm which incorporates each user’s karma rating (a scale of Terrible, Bad, Neutral, Positive, Good, and Excellent), length of membership and randomness. Selected moderators are given special status and 5 mod{eration} points with an expiration window of three days. The moderation status ends when the points have been used in the act of moderating comments or have expired.

The moderation system has been borne out of necessity as the Slashdot community has grown large, bringing the signal-to-noise ratio down and decreasing the satisfaction in reading the raw comment threads. “Flamebait” and “trolls” contribute little more than instigation for starting arguments and fights among the users with typically strong opinions on matters which usually appear on Slashdot. Rob Malda, founder of Slashdot, explains this phenomenon on the Slashdot FAQ:

“One of the unfortunate side-effects of the increasing popularity of Slashdot is that the number of trolls, flame-warriors and all-around lamers increases as well, and it only takes a relatively small number of them to make a lot of noise. Keeping this noise to a minimum is one of the primary goals of the moderation system” (http://slashdot.org/faq/com-mod.shtml#cm500)

Comments are moderated by labeling them as either offtopic, flamebait, troll, redundant, insightful, interesting, informative, funny, overrated or underrated. These adjective labels translate into positive or negative values and are summed to provide the final score. The scoring system for comments is an absolute numeric scale from -1 to 5. Readers can use this scale to filter out comments which have been modded down. While moderators are encouraged to view the comments unfiltered, non-moderators can simply set a viewing threshold which omits the low ranked comments. At times, this creates a disconnect when a highly modded response appears for a hidden comment, however it creates a better overall reading experience (and hidden comments can easily be displayed ad hoc).

Of course, since the moderators are sourced from the same membership pool which is creating the comments to be moderated, controls have been established to limit the influence (or damage) of any one moderator. The small number of mod points provided at a time, short expiration period for them and a somewhat obfuscated moderator selection process make taking advantage of the system by particular users difficult. There is also a metamoderation (m2) system where the larger Slashdot membership can rate the validity of mod points as they have been awarded and can directly influence the karma status of the moderators which awarded them.

Aside from being used to set viewing filters on comment threads, mod points are a direct factor in awarding karma to comment authors. According to the Slashdot FAQ, contributing (good) comments and submitting articles are the primary ways to achieve positive karma, which then increases the chances of being selected as a moderator (for a day, or so). Paraphrased from a story submitted by Slashdot user dkh2: “Post Intelligently, Post Calmly, Post Early, Post Often, Stay On Topic, Be Original, Read It Before You Post, Log In As a Registered User, Read Slashdot Regularly…Come to the party and play.” (http://slashdot.org/faq/com-mod.shtml#cm1900)

In terms of motivators, attaining moderator status provides a sense of all four intrinsic motivators. Being bestowed moderator status confirms a sense of membership in the community, as well as grants a small degree of autonomy. Moderators can recognize competence in other members (as well as affirm their own competence) by modding up particularly good comments, and also display generosity in giving up their limited mod points (although perhaps out of some self-serving drive rather than communal altruism). Modding down poor comments seems similar, but perhaps the generosity is directed toward the community while providing satisfaction at venting ire over undesirable users.

It’s important to note that posting comments is not limited to Slashdot members. “Anonymous cowards” are welcome to post comments on any story. While it seems as though this creates an easy vector to enable trolling and flamebait, the Slashdot chooses to recognize potential utility of anonymous posting:

“We think the ability to post anonymously is important. Sometimes people have important information they want to post, but are afraid to do it if they can be linked to it. Anonymous Coward posting will continue to exist for the foreseeable future.” (http://slashdot.org/faq/com-mod.shtml#cm515)

Anonymous comments begin with zero mod points, so simply adjusting the viewing threshold to 1 and above omits anonymous posts which have not be explicitly modded up and reduces the need for moderators to use up mod points to curtail anonymous flamebait (unless it’s been already modded up for some reason). The stated guidelines for moderators explain Slashdot’s objectives:

“Concentrate more on promoting than on demoting. The real goal here is to find the juicy good stuff and let others read it.…The goal here is to share ideas. To sift through the haystack and find needles. And to keep the children who like to spam Slashdot in check.” (http://slashdot.org/faq/com-mod.shtml#cm600)

While the editors of Slashdot determine which of the submitted stories actually make it to the front page, there has been a conscious decision to keep the commenting system organic. Editors have unlimited moderation points, but comments themselves are not removed. In response to the question “Will you delete my comment?” the resounding answer on the FAQ is:

“No. We believe that discussions in Slashdot are like discussions in real life- you can’t change what you say, you only can attempt to clarify by saying more.…In short, you should think twice before you click that ‘Submit’ button because once you click it, we aren’t going to let you Undo it.” (http://slashdot.org/faq/com-mod.shtml#cm150)

The moderation system has created motivators of its own to keep it self-propagating and self-healing, however the primary motivator among (positive) members of Slashdot is recognition from peers. The tagline for Slashdot is “News for nerds. Stuff that matters.” and there is a real sense that the membership is intelligent, highly competent and well regarded (despite the actual demographics or what can be inferred by the quality of comments). Having a comment deemed as valuable and subsequently modded up provides satisfaction and a sense of accomplishment. There are often choruses of “mod parent up” by non-moderator members imploring peers with moderator status to recognize good comments.

Analysis of larger motivators describing members contributing to a discussion board or comments thread is outside the scope of this examination. However, Slashdot has implemented an interesting self-sustaining system to provide free expression while minimizing inevitable elements which are distracting or possibly damaging to the community. The system is designed to self-moderate and self-select future moderators using recognition of past contributions and actions as indicators of responsibility and competence. A striking element of this system is that lowly moderated comments are not censored but filtered; all contributions are available for viewing by any readers of the site at all times. This is a compelling balance between control and transparency which would be intriguing to experiment with in other contexts.

I’m logging forces experienced by a bicycle {rider} when traveling through an urban environment. Many of these forces go unnoticed by the rider focused in the action of riding.

I’m looking to utilize mobile GPS-enabled handsets as a central part of the sensor platform to take advantage of their increasing ubiquity and a natural tendency for riders to carry a handset at all times. External sensors and hardware may be connected to these handsets to measure specific metrics, such as light and sound levels, air quality, proximity, heart rate, stress, breathing, work, attitude, speed and vibration.

an iPhone application records accelerometer data at anywhere from 1-40Hz (arbitrary rate) which is geocoded using core location (including heading, altitude, and location). every second the most recent sample is uploaded to a server running couchDB. after a ride, the full-resolution file can be e-mailed or uploaded to the server. for the proof-of-concept the data is being plotted on a dynamic Google Map.

in the immediate future i’ll be adding live updating of the map via ajax or some other web sumthin’ dot oh technology.

longer term is to build out similar applications for android and symbian (or maybe just j2me if it’s feasible). i’m also planning on getting back to the ardunio-based sensor packages which started the project.

The Arduino site has a decent walkthrough, but I found some of the steps to be confusing or outdated. Hopefully, this will get you up and running quickly.

Overview

Most of this setup will have to happen once, then starting new projects is typically as easy as duplicating a simple, skeleton project. Getting the paths and configuration options correct for this initial project was the most confusing for me, but I haven’t needed to tinker with them again.

To begin, download Eclipse for C/C++ Developers, or install the C/C++ plugin for your existing Eclipse version (these instructions assume Eclipse 3.5 Galileo). To install an Eclipse plugin:

• Open Eclipse. Navigate to Help > Install New Software…

• Select CDT from the drop down menu. If CDT is not present, then click the add button and paste the link to the plugin update site in the Location field: http://download.eclipse.org/tools/cdt/releases/galileo
• Select Eclipse C/C++ Development Tools, then click “Next >” and “Finish” on the following screen. This may prompt you to restart Eclipse, however select No so we can install the next plugin.

• Return to the Install New Software dialog. Paste in the location for the AVR Plugin and select it and click Next to install. You may have to agree to license terms. http://avr-eclipse.sourceforge.net/updatesite/

• AVR tools. You should install tools appropriate for your platform:
  • Mac OS X: CrossPack (Note: I’m targeting this tutorial for an OS X system)
  • Linux: apt-get install avrdude binutils-avr gcc-avr avr-libc gdb-avr
  • Windows: WinAVR

Setup Eclipse

After the plugins have been installed they need to be configured. Specifically, the paths to the AVR Tools need to be verified.

• Open the Eclipse Preferences: Eclipse > Preferences. Twirl down the AVR category and select “Paths”. Verify that the paths have located the correct AVR toolkit (CrossPack-AVE on OS X). You can edit these fields as necessary.

• Select the AVRDude category. I’ve found that I need to add a Programmer configuration for each Arduino (or USB to Serial adapter) that I use. Please let me know if you can get a better solution working with wildcards.
  • Give this configuration a descriptive name. I’m using “Arduino Diecimila”.
  • Select “Arduino” as the Programmer Hardware
  • Enter the correct port for your Arduino. You can display this by plugging the Arduino into the computer, opening the Terminal then typing: /dev/tty.usbserial then hitting the Tab key. The remainder of the port name should appear.
  • Set the baud rate to 19200
  • Click OK. Then close the preferences.

Create an Arduino project

For the first project you’ll need to manually set up the configuration. I’d *highly* recommend making a basic blink sketch, then duplicate it for each new project. This is what the tutorial will set up. Alternatively, you can try importing this basic project, but may have to set up the paths for your system anyway. Eclipse Blink Sketch

• Go to File -> New -> C Project
• Select Empty Project under AVR Cross Target Application and name the project “Hello_Blink”.
• Click Finish.
• Select the “Hello_Blink” project in the Project Explorer panel, then navigate to Project > Properties menu.
• Twirl down the AVR category and select AVRDude.
  • Select your Arduino from the Programmer configuration menu
• Select Target Hardware at left.
  • Ensure that your Arduino is plugged into your computer via USB (and is powered on).
  • Click “Load from MCU”. This should query the Arduino for it’s type and detect it correctly.
    • If this doesn’t work, typically the Diecimila is an ATmega168 while the Duemilanove is an ATmega328

Each Ardiuno project needs to include the Ardunio core library. This file is compiled automatically by the Arduino IDE. While you could compile it yourself in Eclipse, the easiest way to get it is to build a simple sketch in the Arduino IDE, then copy the “core.a” file into your Eclipse project.

• Start the Arduino IDE
• Open the Blink sketch (File > Examples > Digital > Blink)
• Click the “Verify” button to compile the sketch
• Select Sketch > Show Sketch Folder
• Drag core.a into your Eclipse Hello_Blink project to copy it in.
• Right-click on core.a, select Rename and change it to “libcore.a”

Now we set up the correct building settings:

• Twirl down C/C++ Build and select Settings
• Select “Additional Tools in Toolchain”
• Check the following:
  • Generate HEX for Flash Memory
  • Print Size
  • AVRDude
• Uncheck Generate Extended Listing

• Select AVR Compiler
  • Change the command from avr-gcc to avr-g++
• Select AVR Compiler > Directories
  • Add an Include Path to the header files within the Arduino.app bundle (look at the screenshot)
  • You can right-click on Arduino.app and select Show Package Contents, then navigate to the Contents/Resources/Java/hardware/cores/arduino folder

• Select AVR Compiler > Debugging
  • Set “Generate Debugging info” to “No debugging info”
• Select AVR Compiler
  • Set the Optimization level to Size Optimizations

• Select AVR Assembler > Debugging
  • Set “Generate Debugging info” to “No debugging info”

• Select AVR C Linker > Libraries
  • Add a library named “core”
  • Add a library path. If libcore.a is in the Hello_Blink project folder, the path will be “${workspace_loc:/Hello_Blink}”

Add a new main.c source file

This file will contain your typical Arduino sketch code. There are additional bits of code that are typically added automatically by the Arduino IDE which need to be explicitly written in Eclipse. Feel free to copy the below code.

• Go to File > New… > C Source File. Name this file main.c
• Paste in the following for a basic blink sketch:

#include "WProgram.h"

// prototypes
void blink(int n, int d);

// variables
int ledPin = 13; // LED connected to digital pin 13

void setup(){
Serial.begin(9600);	// opens serial port, sets data rate to 9600 bps
pinMode(ledPin, OUTPUT);      // sets the digital pin as output
Serial.println("------------------");
Serial.println("   hello world");
Serial.println("------------------");
}

void loop(){

  blink(3, 50); // blink led (n times, ms interval)
  delay(1000); // wait 1 sec
}

void blink(int n, int d){
  for (int i=0; i<n; i++) {
    digitalWrite(ledPin, HIGH);
    delay(d/2);
    digitalWrite(ledPin, LOW);
    delay(d/2);
  }
}

int main(void)
{
init();

setup();

for (;;)
loop();

return 0;
}

Building

When you’re ready to build and upload to the Arduino, click the Hammer icon. This should compile your program and then run avrdude, which uploads to the Arduino.

New projects

Duplicating a project in Eclipse is as simple as selecting the source project in the Project Explorer panel, copying it (Edit > Copy) then paste. Eclipse will prompt you for a new name , then will create a new, duplicate project in the workspace.

If you keep the blink sketch intact, you should be able to duplicate it each time and avoid the above setup.

Follow up

You should be all set to work on Arduino projects in Eclipse. Note that Eclipse uses various “perspectives” for different modes and programming languages. Typically, you’ll be working in the C/C++ perspective while working on Arduino code. If you’d like to switch back to the Java perspective for working with Processing, select it from the perspectives at the upper right of the main window, or Navigate to Window > Open Perspective > Other…

I’m creating an animated display, which changes to solid red while braking.

Update: (5:26) It’s been a long night, but it all came together. I wish I could have finished at the floor, but sometimes you need to go to your cave. The bike blinky light is all I could have asked for…it’s obnoxious, it’s animated, it has a handlebar control to switch to a flashing warning mode.

Context:
I got a package of several 5×8 LED matrix displays early in my first semester on clearance from an online electronics wholesaler. I’ve had aspirations to work them into a project ever since but never got around to it.

Several months ago my bicycle taillight stopped working. I took it apart and found a bad solder joint at the LED, so I fixed it and it was happy for several more weeks before acting up again. Now, I’m betting that it’s just a dead battery, but those coin cells can be expensive!

A (more) reasonable person would just get new batteries…performing an effective mental calculation between cost and safety. I say that’s a fool’s errand! Just build a new one, with a rechargeable lithium battery…and while you’re at it upgrade from one LED to 40!

I suppose this is from the “because we can” category. Next steps are to build a weatherproof housing, quick-release mounting, serial control for updating the animation and maybe generative graphics or bluetooth…real-time Twitter feed anyone?

Check out the other ITP30 1-in-1 projects.

I’m logging the acceleration forces at the handlebars of my bicycle while riding through New York City. The body has roughly three contact points with a bicycle, the hands at the handlebars, the “seat” at the saddle, and the feet at the pedals. The downward force of the rider’s weight and pedaling force and the upward forces of the bicycle rolling over uneven ground are distributed over these three points. I was interested to see just what kind of forces are “pushing back” that I may not be aware of, myself lost in the act of simply keeping the bicycle upright and safely navigating through traffic.

To contextualize the raw accelerometer data I also tracking GPS location and eventually geocoding the raw data in software. The bicycle sensors are being transmitted via Bluetooth to a mobile phone and the data is logged with a custom written (but now open-source!) python script. Below is the first draft of the visualization.

I intend to add more sensors to record internal forces to see if there is a physiological response (HR, GSR, breathing, pressure on contact points) to external factors such as speed, traffic / road / weather conditions, time-of-day etc.

Video demonstration of the initial visualization.

This is geocoded accelerometer data from the handlebars of my bicycle on my commute from Greenpoint to ITP in the East Village. The sample at middle of the graph, and the red circle on the path coincide. I realized just now that this is a seam in the Williamsburg Bridge. Neat.

I’ll post proper documentation of the project as it becomes more robust…but I’m excited. Waiting for the heart rate monitor interface to arrive…looking to couple internal status with external events…