Public Lab Research note


Raspberry + NoIR cam + Blue filter

by carolccarvalho | July 15, 2016 14:54 | 1,526 views | 6 comments | #13289 | 1,526 views | 6 comments | #13289 15 Jul 14:54

Read more: stable.publiclab.org/n/13289


What I want to do

Hi!

I'm a student and I'm trying to make an analysis related to plant health. For this, I'm using a drone to fly over some regions, take pictures and turn them into NDVI with the aid of a Raspberry Pi together with PiNoir camera and blue filter that comes with the camera (Roscolux #2007 Storaro Blue).

Earlier I wrote this note: https://publiclab.org/notes/carolccarvalho/02-18-2016/ndvi-from-pinoir-with-a-blue-filter. However, I was adopting a wrong method and thanks to Chris Fastie (@cfastie) I could understand these failures and decided to try this analysis otherwise.

I've also read the @LaPa notes that were of great help to better understand what to do and how to do it.

My attempt and results

As I did not get a way to calibrate the camera during the flight, I wrote a code in shell script that during the flight adjusts random parameters of red and blue using the function "AWB_GAINS" (other parameters such as exposure, shutter speed and ISO for example, are fixed for all images), and then takes pictures in jpg. After taking the pictures I use Matlab to get the NDVI results.

My intention for now is taking over several pictures, separating the red and the blue parameters that provide me a "good" result and then immediately perform another flight with these parameters fixed, to capture more photos. Theoretically with the right fit. I've separated some results I think make sense (obtained on different days). The images are below.

In the image below I used the equation NDVI = (R-B)/(N+B)

resultado.png

resultado_ndvi.png

In the two images below I used the equation NDVI = [(R-B)/(N+B)]+0.25 to "improve" the results. I don't know if it's really right to do this. Can I?

1.jpg

2.jpg

Questions and next steps

Could someone tell me if these results make sense? Am I going in the right direction? As my goal is to take pictures with correct parameters of red and blue, I would like to stop taking pictures with different parameters and fix them so that my results are certain. I suppose the camera misses calibrating. I read that to make a precise calibration is required a spectrometer. Unfortunately I don't have access to a spectrometer, is there any other way to calibrate the camera? Such as using a fixed reference?

Already, thank you very much for your help.


6 Comments

Those NDVI images look reasonable. Deep shadows are not being handled very well so they are assigned NDVI values of 1.0 (magenta) or apparently -1.0 (white). There might not be anything you can do about that except fly when it is bright but without direct sunlight.

It is okay to add a constant to the NDVI values if it is done systematically. The same adjustment must be made for all NDVI images that are to be compared. It is probably better to make the adjustments on the color balance when the photo are being taken so the standard computation of NDVI gives good results, but those adjustments also have to be done systematically for all photos that will be used to make comparable NDVI images.

It is better to get well exposed photos than to get underexposed or overexposed photos and then make adjustments. The underexposed photos above will not have much dynamic range (difference between the brightest and dimmest parts) and the resulting NDVI images could also have a compressed range (that's probably why adding 0.25 helped).

Including a fixed reference (a target of known reflectance) is a standard way to calibrate the process. The standard procedure is to know the reflectance of the target in the NIR and VIS (blue in your case). A shortcut might be to take photos that give what appear to be good NDVI results (healthy plants with NDVI values 0.3 to 0.7, and non plants with NDVI below 0.2). Then use those camera settings and lighting conditions to take photos of some handy targets of different colors. The next time you take photos, include the targets in them and adjust the color balance of the jpegs until the relative brightness in the red and blue channels of pixels in the targets is the same as it was on the day you got good NDVI results. Then use those photos to compute NDVI. You have to find targets that allow the red and blue channels to be well exposed (not too bright or too dim).

It's a lot easier if you have some bright and dim targets that each have the same reflectance for NIR and VIS (or any reflectance as long as you know it) because you can do all the adjustments mathematically. But a less precise method might work.

Chris

Reply to this comment...


Hi Chris, thank you very much for the help!

I'll choose better times and avoid shadows the next flight. And I'll try correcting the exposure too!

I don't know if I understood exactly how to do the color balance, I was left with some doubts... Correct me if I'm wrong please.

1) When you said shortcut, you meant I could for example use a blue led or a blue paint as target? Or do you have another suggestion?

2) Once I get good results with the target in the picture, can I (for the same camera settings and lighting conditions) calibrate using the pixel color of the target from the picture that gave me the best results?

I'm thinking about buying the infragram's Mobius camera. From what I was reading, doing the color balance on this camera is simpler because config file includes information about color balance. Can this color balance be used to make comparable NDVI images? Do you think it's worth changing the camera?

Carol

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


1) When you said shortcut, you meant I could for example use a blue led or a blue paint as target? Or do you have another suggestion?

An LED or other light source can work for calibrating, but it's hard to put it in your outdoors photos, so a colored surface that can be placed in the same light as the plants are in works better. You will need one or more surfaces which reflect both blue and NIR light so that neither channel is too over exposed or underexposed.

2) Once I get good results with the target in the picture, can I (for the same camera settings and lighting conditions) calibrate using the pixel color of the target from the picture that gave me the best results?

Yes, that is the idea. Adjust each new photo (or set of photos) so that the target pixels are the same and then those photos can be compared with each other. I don't know if this has been tried in this simple form, but it should work. You will have to figure out how to do the "adjusting" of the color balance of the photos.

The Mobius Infragram has a nice feature which allows a custom white balance to be set by entering three numbers. Every subsequent photo taken has the brightness of the three color channels adjusted the same way. The closest equivalent I know of for the Pi Cam is a feature of Picamera which allows a custom white balance setting which modifies the result of the automatic white balance algorithm. The setting is called awb_gains and is discussed in some research notes. I think the primary difference is that the automatic white balance system continues to adjust the color balance of each photo, and that balance is then modified by the setting for awb_gains. This could be very effective if all of the photos you were comparing were similar in overall color.

Chris

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Hi Chris, thanks again!

Which are the surfaces that reflect both blue and NIR light? Could you give me some examples?

And about mobius camera, i read that it does a custom white balance and the file "SYSCFG.TXT" contains fixed gains for RGB colors ("Customized White Balance 1 Red Gain=[310],Green Gain=[500],Blue Gain=[700];"). Should I change those gains for different lighting conditions? I still don't understand very well if I have to calibrate the images with mobius camera or if I can just fix the gains at "SYSCFG.TXT".

Carol

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Most surfaces reflect both NIR and visible light. Ideally you would have some targets that reflect various amounts of both and you would know the percent reflectivity for each target in the two wavelength bands you are using (blue and NIR). Then you can use a calibration routine.

I am suggesting a shortcut in which some targets of unknown reflectivity are included in each photo and then the color balance of the photo is adjusted in a program like Photoshop so that the proportion of brightness in the red and blue channels for the targets matches some predetermined level. That level would be determined from photos that give good results for NDVI under similar lighting conditions. So you would have to know what good results for NDVI should be for a particular type of vegetation. This is a very subjective method and has probably never been tried. It was just a suggestion. Some sites with spectral reflectivity of common surfaces are mentioned in this note: https://publiclab.org/notes/nedhorning/11-01-2013/why-a-red-filter-should-work-well-for-ndvi, and in comments on this note: https://publiclab.org/notes/LaPa/03-31-2016/raspberry-noir-cam-sensors-to-detect-water-stress-of-the-plants-during-their-growing.

If you do not use a real calibration routine with known targets and regression analysis, you can adjust the white balance setting of the camera so that the NIR channel (red in your case) is several times brighter than the visible channel (blue in your case) in pixels for healthy foliage. That is the function of the numbers you mentioned for the Mobius camera. This allows photos to be captured that give NDVI results in the proper range, but the particular values depend on lighting, sun angle, and exposure and cannot be compared with other NDVI values unless all those variables are controlled. So using those numbers for a Mobius camera is not calibration, it's a clever hack.

Chris

Reply to this comment...


@carolccarvalho mi nombre es Cristofer, soy de Chile. Llevo 2 años usando raspberry y mas de 1 año aprendiendo sobre NDVI, en la Universidad tuve un curso de vision artificial, el cual me sirvio para programar un codigo que calcula NDVI en python. Acabo de comprar el Phantom 4, tengo una raspberry pi 2B, camara Pi NoIR y filtró que compre en Infragram. Me gustaría que compartiera el resultado de su proyecto para poder ayudarnos. Un saludo. Cristofer

Reply to this comment...


Login to comment.