Public Lab Research note


Registering and processing photos acquired from two cameras

by nedhorning | May 31, 2012 18:16 31 May 18:16 | #2279 | #2279

As people begin mounting near-infrared cameras with visible cameras it will be helpful to automate the processing and registration of the resulting imagery. The open source image processing package Fiji, a distribution of ImageJ, is an excellent tool for doing this.

I created an ImageJ/Fiji macro that will read in any number of paired images that are specified in a text file and provide options to register the images and create the following outputs: NGR image (false-color image with r=near-IR, g=green from visible, and r=red from visible) NDVI image with a user-selected color table applied Floating point NDVI image with actual NDVI values (data range -1 to +1)

The images are clipped to the common area after they are registered and the user can select the output image format.

The processing takes roughly 10-15 seconds per image pair and the image-to-image matching seem to be pretty good. For now I am using a scale invariant feature transform (SIFT) algorithm to select matching points in the two images and then I use those to apply an affine transformation to register the near-IR image to the visible. This should work well if the two cameras are of a similar type, they are mounted so the lenses are reasonably close, and the images are acquired at roughly the same time so the scene composition is nearly identical in both images. Fiji has some additional registration algorithms that will accommodate wrapping but the ones I have used run slowly and warpping shouldn't be necessary if the above constrains are followed.

I'm looking for people to test the macro so if you're using a dual-camera setup you might want to give it a try. For now you'll need to contact me (horning@amnh.org) to get the current version but I'll post it on a web site or maybe github or some place like that if more than a couple people are interested. I will also work on a manual or tutorial if there is interest.


9 Comments

Great note and project. I'm struck by how beautiful this image is Ned. Is the central tree dead or dying?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


I forgot to give the credit for this image to Chris Fastie (sorry Chris). I think the central tree is a pine and they usually have lower NDVI values than the more vigorous broad-leaf trees.

Reply to this comment...


This is a huge advance over the process I have been using. It is possible to manually register two photos (NIR and visible) in Photoshop (as Jeff explains in a YouTube video) and then manually do the calculations in GIMP (as Jeff explains in another video.) That was untenable for large numbers of images. Ned’s macro requires only that you have the free program Fiji installed, and have your near-infrared and visible photo pairs in tiff format. It could also be made to work on jpeg photos, but Ned’s goal was to produce real NDVI values for each pixel. Tiff images avoid the jpeg blurring and probably register with one another better than jpegs, especially when one is taken with a NIR camera and the other a normal camera. Avoiding the jpeg blurring also makes the pixel-level NDVI values more representative of what was actually at that micro-location in the scene. To completely avoid jpeg, photos must be captured in camera RAW or DNG format and then converted to Tiff.

Ned’s macro is also intended for pairs of NIR and visible photos that are almost entirely overlapping. If they are not taken by two similar cameras that are close together and pointing in the same direction, the registration can be poor and the resulting NDVI or NRG values might not be computed with data from the same location (e.g., the same leaf). The resulting NDVI or NRG images could look impressive, but might not be interpretable as scientific results. The current macro might not work very well for images that only partially overlap. That is why we have been working on ways to trigger two cameras on a kite or balloon rig at exactly the same instant.

I hope Ned starts a thread at GitHub for this macro development so more people can test it and maybe contribute to the coding. I keep thinking of new things to have the macro do and I think Ned is trying to avoid me now.

Sara, I’m glad you like my backyard;] The big tree is an eastern white pine that lost its right-hand trunk a couple of years ago. The other trunk is still quite healthy, but not reflecting as much NIR as the surrounding birchs, maples, basswoods and oaks.

Reply to this comment...


I created a GitHub repo for the IamgeJ/FIJI macro. I think you can access it by going to: https://github.com/nedhorning/RegisterPhotos.git

If that doesn't work please let me know. This is my first experience with GitHub and although I have some experience with SVN it might take a while to get the hang of the social aspect of GitHub.

I added a more robust clipping method. My next step is to write a short guide to use the macro and find ways to deal with image-pairs that fail because of an insufficient number of matching points generated by SIFT. An easy option is to implement bUnwarpJ but that take much longer to do the registration. I'm looking at alternatives and any suggestions are welcome.

Reply to this comment...


My first try with Ned's macro for registering and processing photos acquired from ground-test of iPhone/CHDK triggering A490 (IR) and A495 (visible) in RAW DNG format. See:

https://www.flickr.com/photos/coylepdc/sets/72157630049350184/

Reply to this comment...


Hooray, Ned's macro now accepts jpegs, can designate an output directory, and has a user guide (https://github.com/nedhorning/RegisterPhotos.git).

Pat, I learned from the user guide that the "_clipped" image is supposed to be part of the output.

Reply to this comment...


Actually the macro always worked with jpeg images. In ImageJ/Fiji. It has always been able to read any image format recognized by ImageJ/Fiji. Sorry that wasn't clear early on.

I just committed a bunch up updates to the macro based on comments Chris left on GitHub. The macro now has three options for registration algorithms which should make it more robust. I also added some basic error and processing information. Feedback always welcome.

I am trying to keep the user guide up to date with the software updates. If the guide isn't clear or if additional information would be helpful let me know. If you have something to contribute to the coding or documentation that would be great too.

Reply to this comment...


Winds were light, but with Delta kite, was able to fly iPhone/CHDK triggering A490 (IR) and A495 (visible) in jpeg format. This was with original macro. Processed and other images are at: https://www.flickr.com/photos/coylepdc/sets/72157630108709086/

Got updated macro, it registered all pairs. Processed images are at: https://www.flickr.com/photos/coylepdc/sets/72157630053155979/

Reply to this comment...


Hi, ned - do you have the original image pair for the lead image on this page? I'm hoping to use it in the infrared camera kickstarter video if that's all right -- could you share it?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Login to comment.