these are great ideas-- video especially. I've dreamt of creating classic analog tools (like a spring-based tension meter, etc) that could be videoed and turned into real-time sensors. It would be a great way to visualize the math behind measurement. Of course, video analysis is a wicked, processor-intensive problem.

Open CV is trying to address issues like this.

ohh! see simpleCV too

Hey Mathew -- Nice links! You're totally right about video analysis being processor-intensive; Jeff's suggestion to consider audio first made a lot of sense to me.

So last night, Ben Gamari and I hooked up an Arduino running the SoftModem library to the mic jack of my laptop. SoftModem transmits data by representing in binary, then sending sound of two different frequencies to encode "0" and "1". I guess this sort of encoding is called FSK -- http://en.wikipedia.org/wiki/Frequency-shift_keying -- I'm just learning about all this cool stuff now! We wrote some quick code to transmit the number "21" once per second, and then used Audacity to record the sound from the laptop mic. I took a sample of three of these transmissions, and slowed down the third one a lot. The resultant sound is here: http://www.pvos.cc/audio/softmodemTest.mp3

SoftModem translates "21" into its binary form to transmit it; "21" in binary is "010101". SoftModem also adds a initial sound to each transmission so that the receiver knows to expect something. So what you hear is first, that long initial sound; then, the two chirps corresponding to "21" being broadcast at regular speed; then the third chirp has been slowed down a lot, so you can hear the initial handshake sound, then the "010101" pattern.

Now a next step might be to find a way to decode this using HTML5 / a browser, so that any laptop / smartphone with an audio jack + modern browser can get data from an Arduino. FUN!

The audio jack seems like a decent stop-gap. Long term, however, I would highly suggest trying to work with the likes of Mozilla to get some kind of browser USB access (and maybe Bluetooth as well) codified. Mozilla is doing a good bit of related work already for Firefox OS, and most new phones should be able to play host to USB devices thanks to USB OTG. There will come a time when high-speed data transfer to scientific web applications will be invaluable, so I say let's get started on it now.

Great progress, Don!

http://phenomnomnominal.github.com/docs/tuner.html

pitch detection!

Jeff -- WHERE did you find that link?! I think you just solved the problem!

Cov -- I totally agree, and actually it took me a while to convince Ben that spending any time at all on this audio comm protocol was a good idea :) Firefox OS is really exciting; once web apps can access USB, then we're off to the races. Fully featured scientific web applications are going to require high-speed data transfer, and that's really where we should be heading.

The reasons (such as they are) that I'd give for justifying this sort of tinkering in the meantime are: 1) If we can get audio comm working, we can start prototyping and using the relevant web app software now. Having working prototypes will help speed and motivate the web app development. 2) There are currently lots of high school students in classrooms with desktop computers for which even the teacher doesn't have permission to install software. Audio comm + web apps could circumvent this problem; 3) This sort of hack demonstrates that communication doesn't have to occur via industrially-established protocols / equipment. I.e., end users can more readily recognize that they don't have to be at the mercy of the telecom industry's priorities / agendas for how we use the hardware we ostensibly own; 4) I felt like Jodie Foster in "Contact" when we saw the "010101" in our Audacity-captured audio signal. So #4 is "You can feel like Jodie Foster in 'Contact'".

All that as an aside, I totally agree with you -- we also should work with Mozilla to push for browser USB access. It's going to be so cool when that happens.

I'm still trying to wrap my mind around how awesome it is that these web apps allow you to "right-click view source". Truly open source, copy-paste-modify. Crazy cool.

I'd been looking into decoding audio for an unrelated, semi-silly project some time ago: https://github.com/jywarren/hellschreiber-js

and by the way, if you can't figure out tone decoding, there's no shame in using full-on, full-off noise and measuring volume -- you can make short/long tones and basically do Morse code. Might not be a bad idea for a first implementation/proof-of-concept!

Jeff --

-- Hellschreiber.js: Holy (sic) guacamole! A few years ago, a physicist presented a Darpa-funded project at UMass. Darpa's request was: build us a system that would allow for long-distance communication after an EMP attack -- i.e., if all one's electronics were kaput. His solution involved making lots of little droplets of chemicals that underwent oscillatory chemical reactions that could be controlled, so that the material would show as a dark dot, or a light dot -- basically, a system could communicate in morse code visually, as long as one had a telescope and an array of these things (I guess). I need to wrap my head around the hellschreiber.js project. I tend to conflate things I shouldn't conflate, but, prima facie (from site explaining the technology), it seems related to me.

-- Morse code encoding / decoding is a great idea. A lot easier for me to get a handle on, conceptually, than implementing an audio version of an RS-232 protocol or whatnot. Also: engaging for high school students to see two computers communicating via an easy-to-understand alphabet. And eminently hackable.

-- Heck, if you get a browser to understand morse code via audio, you could even communicate with the browser by tapping out the signal yourself! How cool would that be as a demo of physical computing / communication, for students. And, jeez, I just want to be able to do that.

-- Just saw the irkit. Brain. Exploding. Stop with the cool ideas, for a sec? Whimper.

I find it hilarious that DARPA would pay someone to reconceptualize optical semaphore, which is more than 200 years old.

awesome work on the audio jack! keep it up.

Mathew,

Thanks for the optical semaphore link! This really sent me down a wonderful rabbit hole of fun ideas. I had no idea that Hooke gave a seminal lecture on semaphore tech whilst he was in the midst of characterizing how springs respond to stretching. May those glorious times of working on projects in several different 'fields' return! (I guess Hooke was working before there were these silly 'field' definitions ...)

One link I landed upon was this fellow at CalArts who seems to be wonderfully obsessed with data transmission schemes. Really helpful historical review of various methods: -- history of character codes: http://www.wps.com/projects/codes/ -- "bits, bauds, and modulation rates": http://www.wps.com/J/bits-bauds-modulation-rates.html

I was talking to a friend a day or so ago, and he seconded Jeff's suggestion of simply looking at the signal amplitude as a way of conveying bits, rather than processing frequencies. This certainly seems like a simpler scheme. I'm not yet hip enough to all of this to see what advantages frequency might have -- I'm guessing it allows one to avoid having to establish a baseline amplitude when interpreting a signal via amplitude? But anyway, might as well start with amplitude, as it's so much easier (I think).

Then I thought this morning (I think this just goes back to one of Jeff's original ideas): if one is simply measuring amplitude of a visual signal, is that relatively lightweight in terms of processing? Certainly less processor-intensive than, e.g., reading a QR code. And the code for interpreting e.g. a blinking LED wouldn't look so different from interpreting a high/low audio signal ...

The semaphore connection is, again, really, really cool. Leads one to imagine all sorts of alternative communication schemes, involving everything from purely mechanical lens + shutter + ink recording, to laser modulation at a distance ... might be a nice way of getting around FCC rules on radio broadcasts, when communicating with an airborne platform (camera on a kite, Raspberry Pi on a balloon) ... could a light sensor on the ground interpret flashes from a strobe light on a balloon? How much power do strobe lights require? Too much fun ...

A friend just showed me this TED Talk -- "wireless data from every light bulb" -- the fellow makes some nice arguments for using visible light to stream data wirelessly, and includes a nice demo:

http://www.ted.com/talks/harald_haas_wireless_data_from_every_light_bulb.html

One more link for this, semi-related: https://github.com/westonruter/html5-audio-read-along

Oh, wow! "Audio-read-along" worked in my browser (latest stable Chrome, on Ubuntu) -- and apparently should work on latest stable versions of Opera, Chrome, Firefox ... okay, just gotta dig in and figure out how it works, now. Exciting.