My contribution to the project was the web voting interface and database backend as well as a simple API for communicating with it. I wrote the application using the Sinatra microframework for Ruby and utilized couchrest to communicate with a CouchDB database. A control panel interface enabled an operator to update the voting interface status messages and to open the next event for voting. A display page mocked up the live video with Fly and Pie status bars for viewers to watch.

Although the system is very rudimentary, all the pieces are in place for a live voting situation. During the class demo however, the server was crippled by only a dozen simultaneous connections, and become sluggish enough to be unusable. I believe that the problem stemmed from each of the clients polling the database, and couchrest seems to update a view each time it’s requested. CouchDB is quite fast at reads, but far slower at writes, and I think these things combined to make a big problem.

I think that performance could be greatly enhanced if status polling were eliminated, or at least read from static file that was generated by the control application or server when data needed to be refreshed.

Live voting is a tested and successful model, and the class seemed to enjoy the real-time updates at the beginning while the system was responsive. The show itself was successful largely thanks to Gordie and Lisa Maria’s performances (and willingness to take a pie to their faces from Jong Yoon).

Below is a short python script to parse a Foursquare KML file into a simple CSV file. It outputs the check-in name, description, timestamp and location (as lat, lon). The Foursquare KML feed is available at the Feeds page on their site.This script relies on BeautifulSoup and, of course, python. Give the script the filename of the KML file you want to parse and it will output out.csv in the current directory. eg.

$ python parse_kml.py MyFoursquareFeed.kml

Here’s the file: parse_kml

# parse each file from the photo collection and export data into CSV.
# will need: os.listdir(path)
import sys
import os
import codecs
import csv
from BeautifulSoup import BeautifulStoneSoup

# get the file list:
if len(sys.argv) > 1:
    dir = sys.argv[1]
else:
    dir = os.getcwd()

file = dir

# create the output dictionary
outputData = []

# sanity checking, only work on kml files
if file.endswith('.kml') == 0: sys.exit(-1)

print "Reading file: "+file

fh = codecs.open(file,'r',"utf-8")
html = fh.read()
fh.close()

soup = BeautifulStoneSoup(html)
#print soup.prettify()

# create a new dictionary for the current image's data
imageData = dict();

# get the image data:
dataTable = soup.findAll('placemark')
for i in dataTable:
    row = i.contents

    # add the current data to the dict
    imageData = {}
    imageData['Name'] = row[0].contents[0].string.encode("ascii","ignore")
    imageData['Description'] = row[1].contents[0].string.encode("ascii","ignore")
    imageData['Time'] = row[3].contents[0].string.encode("ascii","ignore")
    coord = row[5].coordinates.contents[0].string.encode("ascii","ignore")
    imageData['Lon'] = coord.split(',')[0]
    imageData['Lat'] = coord.split(',')[1]

    # add this image's data to the list
    outputData.append(imageData)

#print outputData

# create the output file
out = codecs.open(os.getcwd() + "/out.csv", 'w',"utf-8")
firstRun = 1

print "Writing output file: "+ out.name
try:
    fieldnames = sorted(outputData[0].keys())
    fieldnames.reverse()
    writer = csv.DictWriter(out,dialect='excel', fieldnames=fieldnames, extrasaction='ignore', quoting=csv.QUOTE_NONNUMERIC)
    headers = dict( (n,n) for n in fieldnames )
    writer.writerow(headers)

    for row in outputData:
         writer.writerow(row)

finally:
    out.close()

The app arranges illustrations of animals wearing lettered t-shirts to create user-supplied messages. Kara Schlindwein created the illustrations as part of her project for the 6th Annual Fun-a-Day project in Philadelphia, and I wrote the first draft of the app while sitting in a chair at the show a couple of weeks ago, still nursing my broken ankle.

I’ve come to really like working with Sinatra / HAML / SASS – once I had a basic comprehension of how the components fit together I could see how to organize the app without getting mired in the code. Making changes to the app has been pleasantly easy. Deployment via Passenger (mod_rails) on Apache was simple enough my my local and development servers (after reading sample virtual host configurations) and setting up a subdomain on Dreamhost has been fine, too. Everything is coordinated via git, so pushing changes around has been simple.

I first heard about Sinatra (and couchdb for that matter) in a presentation by Daniel Tsadock at ITP last year and I’ve kept it in the back of my mind as something I wanted to try. I’m just now getting to development of my thesis project website and decided to use Sinatra for it, although I had a disappointing experience with the performance of another Sinatra + couchrest app I wrote for the Live Experimental Interactive Television class this past week. My VPS development server ground to a halt when a dozen classmates attempted to use the app at the same time. I’m pretty sure that my code was somewhat inefficient (with each client polling couchdb through an api provided via Sinatra routes), however I was hoping for usability beyond 12 simultaneous connections. I’m still optimistic though that it will work, and writing an app with the framework and view templates is fast and actually pretty fun.

The project page is: Pocket OCR

The source code is available at github: http://github.com/rcarlsen/Pocket-OCR

There is certainly a lot of improvement to be made. Automatic color correction. Page layout recognition. Perspective correction…the list could go on. The code is there, so…fork away!

(the thumbnail is a bit tongue-in-cheek…but honest. good conversion requires a good source image: well-lit, macro, focused and tightly cropped seems best)

i finally got around to building a proof of concept implementation of tesseract-ocr for the iPhone. months ago, i documented the steps which helped to get the library cross-compiled for the iPhone’s ARM processor, and how to build a fat library for use with the simulator as well. several folks have helped immensely in noting how to actually run the engine in obj-c++. thanks to everyone who has commented so far.

anyway, below is a short video of the POC in action. the basic workflow is: select image from photo library or camera, crop tightly on the box of text you’d like to convert, wait while it processes, select / copy or email text.

there are loads of improvements which could be implemented (image histogram adjustment, rotation / perspective correction, automatic text box/layout detection, content detection – dates, links, contact information…) but this is a nice point to stop and document.

i realize that there are several OCR applications available for the iPhone, including a few which also run the engine on the device rather than handing it off to a web service. this started as an educational project on cross-compiling, and to fill a personal want for a handheld OCR app of my own. for these reasons, i’m going to open-source the entire app. look for it after this semester ends when i’ll have some more time to properly document the code. in the meantime, enjoy these code snippets demonstrating how to initialize the engine and process an image.

Initialize the engine:

    NSString *dataPath = [[self applicationDocumentsDirectory] stringByAppendingPathComponent:@"tessdata"];
    /*
     Set up the data in the docs dir
     want to copy the data to the documents folder if it doesn't already exist
     */
    NSFileManager *fileManager = [NSFileManager defaultManager];
    // If the expected store doesn't exist, copy the default store.
    if (![fileManager fileExistsAtPath:dataPath]) {
        // get the path to the app bundle (with the tessdata dir)
        NSString *bundlePath = [[NSBundle mainBundle] bundlePath];
        NSString *tessdataPath = [bundlePath stringByAppendingPathComponent:@"tessdata"];
        if (tessdataPath) {
            [fileManager copyItemAtPath:tessdataPath toPath:dataPath error:NULL];
        }
    }

    NSString *dataPathWithSlash = [[self applicationDocumentsDirectory] stringByAppendingString:@"/"];
    setenv("TESSDATA_PREFIX", [dataPathWithSlash UTF8String], 1);

    // init the tesseract engine.
    tess = new TessBaseAPI();

    tess->SimpleInit([dataPath cStringUsingEncoding:NSUTF8StringEncoding],  // Path to tessdata-no ending /.
                     "eng",  // ISO 639-3 string or NULL.
                     false);

Process an image. This should be threaded as it’s a heavy process:

    CGSize imageSize = [uiImage size];
    double bytes_per_line	= CGImageGetBytesPerRow([uiImage CGImage]);
    double bytes_per_pixel	= CGImageGetBitsPerPixel([uiImage CGImage]) / 8.0;

    CFDataRef data = CGDataProviderCopyData(CGImageGetDataProvider([uiImage CGImage]));
    const UInt8 *imageData = CFDataGetBytePtr(data);

    // this could take a while. maybe needs to happen asynchronously.
    char* text = tess->TesseractRect(imageData,
                                     bytes_per_pixel,
                                     bytes_per_line,
                                     0, 0,
                                     imageSize.width, imageSize.height);

    // Do something useful with the text!
    NSLog(@"Converted text: %@",[NSString stringWithCString:text encoding:NSUTF8StringEncoding]);

    delete[] text;

Enjoy the video!

iTunes also seems to be entirely unhelpful when something goes wrong with the “unknown error” message. What could it be? Code sign error? Unprovisioned device? Incorrect OS target? Mismatch in the app and mobileprovision file? It’s all a mystery in the iTunes universe.

Despite the inconvenience involved with having to generate a new ad hoc mobileprovision profile each time I add a new beta tester device, requiring a recompile of the beta app, it’s nice to finally get an app out to other non-development devices.

The ad hoc distribution process is documented well (enough) on Apple’s dev portal and on several other blogs, but my real stumbling block was navigating the Xcode interface to ensure that the proper code signing profile was being included in the binary (hint, look at the verbose build output for “embedded.mobileprovision” and ensure that the mobileprovision ID matches the correct version.)

I’ve also been getting trouble with Xcode not building using the correct provisioning, despite updating the target info and cleaning out the build folder. A restart of the program usually gets it sorted.

The next step will be to set up a Distribution build for the App Store submission. We’ll see how that goes in the end…

there was no improvement despite several rounds of optimizing the openGL drawing code and plugging several memory leaks.

finally, in frustration i bumped the explicit frame rate declaration from the default 60 fps to 120 fps. i realize that ofSetFrameRate(60) is merely the upper limit of the frame rate, and that the hardware won’t actually go faster than it can handle, however this immediately improved performance on the 3GS and the first gen is about the same as it was. further improvement was had with a declared frame rate of 240 fps.

i haven’t had a chance to look into the underlying issue, but i believe that oF is using NSTimer under the hood to trigger a scheduled update() and draw(). has there been some change in the SDK there?

there are hundreds (if not thousands) of these messages, and the object allocation graph trends steadily upwards. when i disable the portion of the method which updates the center property the allocations stop and the memory usage goes flat. this also only occurs on the device…the simulator does not exhibit this behavior.

i’m using UIImageView directly, without subclassing, and it contains data from a small PNG file. i’ve patched up several other leaks successfully today, and really wanted to nail this one too.

I believe that archiving objects require methods inherited from NSObject, which we don’t have in openFrameworks’ ofSimpleApp. Core Data seems like overkill, so I looked into using plist files.

There are likely better ways to do this, but this ad-hoc solution works wonderfully for a small app I’m working on, and only requires a bit of Objective-C code that could likely be moved up into a nice wrapper class. However, since the question was asked I’d just like to get it out there before working on a more elegant approach.

The easiest way to store state is with NSUserDefaults. The following returns a pointer to the shared user defaults object then writes several values to it. Adding data to the object will get cached in memory and eventually written to disk (or upon receiving a synchronize message). The key is just a NSString, which can be arbitrarily named, but you’ll need to use identical values to retrieve the data later.

NSUserDefaults *savedData = [NSUserDefaults standardUserDefaults];

[savedData setInteger:score forKey:@"score"];
[savedData setInteger:pointsAvail forKey:@"pointsAvail"];
[savedData setInteger:timeRemaining forKey:@"time"];
[savedData synchronize];

The savedData object should be autoreleased. Plists can handle basic data types such as integer, float, bool and double. Look at the documentation for specific messages, but it’s usually like setFloat:forKey: or setBool:forKey:. It can also handle some NSObjects such as NSString, NSArray and NSDictionary. The selector for those is setObject:forKey:.

Reading back is just as simple. If a key doesn’t exist 0 is returned for a number, and nil for other strings, arrays and dictionaries. In the below example score, pointsAvail and timeRemaining are instance variables of testApp.

NSUserDefaults *savedData = [NSUserDefaults standardUserDefaults];

score = [savedData integerForKey:@"score"];
pointsAvail = [savedData integerForKey:@"pointsAvail"];
timeRemaining = [savedData integerForKey:@"time"];

To get this all working as expected, retrieve the defaults in setup(), and be sure to do some sanity checking if the values aren’t found (ie. if this is the first run of the app). Because we’re mixing Objective-C and C++ in testApp, be sure to rename main.cpp to main.mm, testApp.cpp to testApp.mm and add “#import <UIKit/UIKit.h>” to the top of testApp.mm.

A code snippet from setup():

// test for previous state
NSUserDefaults *savedData = [NSUserDefaults standardUserDefaults];
int hideInstructions = [savedData integerForKey:@"hideInstructions"];

// if the key "hideInstructions" doesn't exist, then it will return 0
if(hideInstructions) {
// we've must have run at least once before and written "hideInstructions"
// restoreGame will read in the rest of the saved values:
restoreGame();
} else {
// assume first run
[savedData setInteger:1 forKey:@"hideInstructions"];
showInstructions();
}

openFrameworks supports an exit() method which gets called dutifully on the iPhone when the application will quit. This is a fine place to write out data to the userDefaults. The challenge as a programmer will be to determine the appropriate data to save and restore. In my simple game, I needed to save the game state for score and time, but also the position and basic state of the widgets on screen. With a larger game I’d look into writing methods into my objects cough up their own important data and then request it when the app is exiting. Keep in mind that there is a narrow window of time that the iPhone OS provides between receiving a signal to quit and when it will forcefully quit your app (like a few seconds) so saving data needs to happen quickly.

This method is purposefully rudimentary, but I think it works well for this particular use.

Feel free to let me know how I can do things better, or just tinker with the code directly at github: ofxiPhoneWrappers

Check out the example project. This is a bit of a kludge. I would like to figure out how to avoid having to declare main and testApp as mixed source files (.mm). It works for now, however.

Here are the notes on using the class:

// ofxiPhoneAlertView:
// setup the list of buttons
vector<string> otherButtons;
otherButtons.push_back("Button 1");
otherButtons.push_back("Button 2");
// the args are: title, message, cancel button title ("" == omitted), vector of other button titles
ofxiPhoneAlertView *alert = new ofxiPhoneAlertView("Title,"Here is the alert message.","",otherButtons);
// display the alert view
alert->show();

// I'm using the following methods in update()
// test if the alert is visible (bool):
alert->isVisible();

// test if the alert has been dismissed:
alert->isDismissed();

// get the selected button (int):
alert->getSelectedButton();

// if you use several alerts, you can distinguish between them via their title:
alert->getTitle();

// example logic:
switch(alert->getSelectedButton()){
case -1:
// cancel, or no selection
printf("alert = -1");
break;
case 0:
// first, non-cancel button
printf("alert = 0");
break;
case 1:
// next button. will be the second button after Cancel
printf("alert = 1, ok");
break;
}

ofxiPhoneAlertView.h

/*
ofxiPhoneAlertView.h

Created on 6/10/09.
Copyright 2009 Robert Carlsen | robertcarlsen.net

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.

*/

#import <UIKit/UIKit.h>
#import "ofMain.h"

@interface ofxiPhoneAlertViewDelegate : UIAlertView <UIAlertViewDelegate>
{
UIAlertView*			_alertView;
int                     selectedButton;
bool                    dismissed;
}

- (id)initWithTitle:(NSString *)title message:(NSString *)message cancelButtonTitle:(NSString *)cancelButtonTitle otherButtonTitles:(NSArray *)otherButtonTitles;

-(int)getButton;
-(bool)isDismissed;
-(string)getTitle;

@end

class ofxiPhoneAlertView
{
public:
ofxiPhoneAlertView(string title, string message, string cancelButtonTitle, vector<string> otherButtonTitles);
~ofxiPhoneAlertView();

void show();
int getSelectedButton();
bool isVisible();
bool isDismissed();
string getTitle();

protected:
ofxiPhoneAlertViewDelegate *alertViewDelegate;
};

ofxiPhoneAlertView.mm

/*
ofxiPhoneAlertView.mm

Created on 6/10/09.
Copyright 2009 Robert Carlsen | robertcarlsen.net

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
*/

#include "ofxiPhoneAlertView.h"

// c++ class methods
ofxiPhoneAlertView::ofxiPhoneAlertView(string _title,
string _message,
string _cancelButtonTitle,
vector<string> _otherButtonTitles){
NSString * title = [[[NSString alloc] initWithCString: _title.c_str()] autorelease];
NSString * message = [[[NSString alloc] initWithCString: _message.c_str()] autorelease];

// nil value will omit button
NSString * cancelButtonTitle;
if(_cancelButtonTitle == "")
cancelButtonTitle = nil;
else
cancelButtonTitle = [[[NSString alloc] initWithCString: _cancelButtonTitle.c_str()] autorelease];

// the other buttons are an array of strings
NSMutableArray *otherButtons = [NSMutableArray array];
for(int i=0;i<_otherButtonTitles.size();i++){
[otherButtons addObject:[[[NSString alloc] initWithCString: _otherButtonTitles[i].c_str()] autorelease]];
}

alertViewDelegate = [[ofxiPhoneAlertViewDelegate alloc] initWithTitle:title
message:message
cancelButtonTitle:cancelButtonTitle
otherButtonTitles:otherButtons];

}

ofxiPhoneAlertView::~ofxiPhoneAlertView(){
[alertViewDelegate release];
}

void ofxiPhoneAlertView::show(){
[alertViewDelegate show];
}

int ofxiPhoneAlertView::getSelectedButton(){
return [alertViewDelegate getButton];
}

bool ofxiPhoneAlertView::isVisible(){
return [alertViewDelegate isVisible];
}

bool ofxiPhoneAlertView::isDismissed(){
return [alertViewDelegate isDismissed];
}

string ofxiPhoneAlertView::getTitle(){
return [alertViewDelegate getTitle];
}

// obj-c implementation
@implementation ofxiPhoneAlertViewDelegate

- (id)initWithTitle:(NSString *)title message:(NSString *)message cancelButtonTitle:(NSString *)cancelButtonTitle otherButtonTitles:(NSArray *)otherButtonTitles
{
if (self = [super initWithTitle:title
message:message
delegate:self
cancelButtonTitle:cancelButtonTitle
otherButtonTitles:nil ]){
_alertView = [[ofxiPhoneAlertViewDelegate alloc] initWithTitle:title
message:message
delegate:self
cancelButtonTitle:cancelButtonTitle
otherButtonTitles:nil ];
for(NSString *s in otherButtonTitles){
[_alertView addButtonWithTitle:s];
[self addButtonWithTitle:s];
}

dismissed = false;
//self = [_alertView copy];
}
return self;
}

-(int)getButton{
return selectedButton;
}

-(bool)isDismissed{
return dismissed;
}

-(string)getTitle{
// return the title of the alert. to differentiate between separate alert views
return [[self title] UTF8String];
}

/*
- (void)dismissWithClickedButtonIndex:(NSInteger)buttonIndex animated:(BOOL)animated;
{

}
*/
// Called when a button is clicked. The view will be automatically dismissed after this call returns
- (void)alertView:(UIAlertView *)alertView clickedButtonAtIndex:(NSInteger)buttonIndex
{
selectedButton = buttonIndex;
dismissed = true;
NSLog(@"Selected button: %d",selectedButton);
}

/*
// Called when we cancel a view (eg. the user clicks the Home button). This is not called when the user clicks the cancel button.
// If not defined in the delegate, we simulate a click in the cancel button
- (void)alertViewCancel:(UIAlertView *)alertView;

- (void)willPresentAlertView:(UIAlertView *)alertView;  // before animation and showing view
- (void)didPresentAlertView:(UIAlertView *)alertView;  // after animation

- (void)alertView:(UIAlertView *)alertView willDismissWithButtonIndex:(NSInteger)buttonIndex; // before animation and hiding view
- (void)alertView:(UIAlertView *)alertView didDismissWithButtonIndex:(NSInteger)buttonIndex;  // after animation
*/

-(void) dealloc{
[_alertView release];
[super dealloc];
}

@end