Mobile Logger – DB export and zip compression

db-zipGot two new features rolled into Mobile Logger at once…raw database export and zip compression. This sounds exciting, right?!

Actually, it’s pretty cool and useful. Some of these new export formats (GC and GPX, I’m looking at you) are XML-based and thus quite verbose. What was a manageably small CSV file or JSON string has become inflated much larger from all the tagging in these formats. Exporting a log in one of these formats means having send a doubly large file (approx. 300kB -> 600kB). However, all the repeated tags makes them great candidates for zip compression. Those logs are about 70kB afterwards. So there’s that.

db-settings

Secondly, I have been thinking of making it easy to get the raw database off the device. It’s a good idea for backup and since it’s sqlite, advanced tinkerers could run custom queries and maybe figure out neat things…like where do I ride fastest, generally? To that end I’ve added a row in the settings panel to e-mail the raw database directly. This is what really pushed the compression initiative forward. The db on my device is 14MB, which is difficult for the phone to e-mail (lots of memory warnings and it took forever to send). Compressing the file makes it about 3MB, which is still large but quick enough over WiFi.

However, making the raw database available does expose a pretty big shortcut that I took when first putting the app together. All the sensor/log data is stored in the db as big JSON strings. This made it really easy to save locally and send to couchDB on the fly while a log was being recorded. Also, it didn’t matter what data I was storing – couchDB is schema-less and it was nice to not have to define a local schema. The huge downside is that every export feature parses the JSON strings and then formats each datum as necessary…it takes a bit of work on the phone’s part. Further, it makes it very difficult to use SQL queries to inspect the log data…aggregates, relationships…all that would have to be parsed into memory first. Ugh. Not happening.

So, on the horizon I suppose is gutting the data model and implementing the schema properly. The sensors on the iPhone aren’t changing – at least until we get to connect to external sensors ;) .

As always, code is on github.

Leave a Reply