Oh that's right, the photos have to come back from the cameras into the USB ports so it would be hard to share one port.

Shutter sync won't be that important for ground based plant cam uses, but aerial work usually requires pretty good sync. This note has a link to a nice timer with a big 1/100 second display which is great for documenting the sync you are getting. Unfortunately the timer is Windows only.

That's nuts that the Pi operating system inhabits only 2 GB.

Sadly, the v4l2 upgrade did not fix the USB bandwidth problem for dual webcams; this issue probably isn't a bug but really a limitation of the cheap webcam hardware with the expected usage case of the drivers (full video streaming). The webcams have no on-board mjpeg encoder, which is one method of lowering the bandwidth requirements. The reason that Don's laptop works must be that his USB ports are on separate conrollers so are not fighting for the bandwidth - I don't think this is the case with the RPi's two USB ports. Short of finding (or writing!) a new driver that allows low FPS high res. capture, we are left with a semi-satisfactory work-around: force the stream data structure to unload after taking the capture, but before loading the other camera's stream; unfortunately, this introduces a ~0.5 second lag on my fast computer, which may be worse on the RPi. (Since this is one program, the timing can be done in situ by requesting timestamps from the OS [easier than it sounds].) The lag would only be about 0.005 seconds or less if we didn't have to reload the stream every time. I am going to muck around a bit more with the low-level stuff and see if we can't do better.

For those of you who really want to know the details of the USB bandwidth problem, the UVC driver site has this FAQ item.

Wow, it sounds like you are operating right on the edge of the system's capabilities. That problem is related to streaming video, and for the IRcam you need only transfer a single frame from each camera. But I guess the problem is that with little or no RAM for buffering, the cameras must transfer the photo as soon as it is captured. And for synchronous photos, that means two full frames trying to get through one USB controller at the same time and maxing out the limit. Is one option an additional USB controller, or would that be more trouble and expense than just using two Pis? An upgrade to the cameras might help if the new ones had more RAM or MJPEG compression, but that would probably cost more than a second Pi.

Maybe the land-based Plant Cam can have asychronous capture and a more expensive option is needed for aerial work.

Chris, your comments are right on the "money" (pun intended). It looks like the best we can do with these budget cams, short of writing a new driver (if even possible given the hardware constraints), is have two modes: high res (1280x1024)/high lag (~0.5 sec); low res (320x240)/low lag(~0.005sec).

Is one option an additional USB controller, or would that be more trouble and expense than just using two Pis?

I'm not sure if that is possible and cheaper, but maybe we could do one USB camera and one ethernet camera, since on the Model B they are separate hardware controllers. Two RPi's is also a realistic option though synchronization would be tricky, but the money might be better spent on fancier cameras.

Maybe the land-based Plant Cam can have asychronous capture and a more expensive option is needed for aerial work.

I'm mostly in agreement, but I'm going to toss out a hypothetical to the experts: For the aerial mapping application, maybe the lag isn't as bad as it seems -- remember that the scene isn't moving, only the perspective is. If we take enough shots to get coverage, then maybe the images can be affine-aligned, then stitched into two sheets, IR and VIS, then overlaid. Another problem might be motion blur, but as you know, this could be algorithmically corrected too. We could call it an "IR/VIS point and shoot (and stitch) camera".

You're right that for aerial work a 0.5 second lag is not terminal. The whole rig will move some, but the photos from the two cameras will still overlap a lot. They can be registered for making NRG or NDVI, but the overlap will sometimes be only 60-80%, so the cropped composite will generally be much smaller than a full frame. So more photo pairs will be needed to get complete coverage of the scene, more photo pairs will have to be registered, normalized, and combined into multispectral composites, and there will be more composite images to stitch together into multispectral maps, which will have more stitching seams and artifacts. But it will still work.

In fact, it provides some interesting information. If the lag is constant, photo pairs that do not overlap a lot will have been taken while the rig was moving a lot. Those photos will have more motion blur and probably be less vertical than photos from pairs that overlap a lot. So a solution is to take a lot of pairs and throw out the pairs that overlap less than x. The result will be better quality than your average pair of perfectly synchronous kite photos.

Stitching all the VIS into one map and all the NIR into another and then rectifying the maps is a non starter. The rectification will be lousy. And the stitching process is manual, so you only want to do it once. Or three times if you want VIS, NRG, and NDVI. Or once if somebody figures out how to use the information in the first manually stitched map to create the other two. You would have to record all of the manual placement and affine transforms for each image. No problemo, Baby.

Reposting a comment that somehow slipped through the cracks:

Comment by kscottz: Hi Guys, I am one of the SimpleCV devs and I noticed your post. I think we work together to fix your issues and create a better library all around. Can you either toss a post up on our help forum (http://help.simplecv.org [1]) or file an issue github and we'll take care of it just as soon as we can.

Hi kscottz, We are trying to do image registration (affine alignment) with shots taken from slightly different viewpoints. The issue I believe was already raised a while ago in this thread on your help forum. In the reply by "Rishi Mukherjee" he gives a link to a patch that he wrote which makes a change to the homography matrix of the method ImageClass.Image.findKeypointMatch in this github pull request. I cannot vouch for the sanity of this patch, but I can confirm that the homography matrix that it exports is compatible with the function cv2.warpPerspective; whereas, the method in the latest pip install SimpleCV (v1.3) exports a homography matrix that gives an incorrect/incompatible transformation that is rotated and somewhat distorted. I don't know that much about image manipulation, but maybe the problem has something to do with the different conventions used?

follow the technical Python code discussion here: https://github.com/ingenuitas/SimpleCV/issues/320

In pursuing the image registration problem, some more bugs were found in SimpleCV that cause some colorspace image formats to get accidently inverted during geometrical transformations. (Really the fault occurs because of row-major vs. column-major array formatting contentions - a tired old example of computational conventions that don't "really matter" but no one can agree on!)

Don't get me wrong, I am glad that the SimpleCV project exsists and I will continue using it as long as the bug fixes keep coming. Its because of these silly format incompatibilities and the awful legacy of C/C++ syntax that Python OpenCV library is not already simple!

For technical code discussion these issues can be tracked at: https://github.com/ingenuitas/SimpleCV/issues/339