Public Lab Research note


TetraPi: a well characterized multispectral camera

by khufkens | April 24, 2015 15:54 24 Apr 15:54 | #11767 | #11767

khufkens was awarded the Basic Barnstar by juvinski for their work in this research note.


khufkens was awarded the Excessive Enthusiasm Barnstar by jarrarte for their work in this research note.


Democratizing multispectral imaging

Motivation and goals

Many of the projects on the Public Lab deal with infragrams, NDVI or NRG images to measure vegetation health. Although the images produced do discriminate between healthy and diseased or stressed vegetation, the imageging pipeline is not well characterized. Furthermore, commercial companies offering well defined multipsectral cameras (> 2 channels) do this at a steep price tag (~$5000).

Both the lack of a of a well characterized image sensor, which makes quantitative research impossible (by using inverse modeling using amongst others PROSAIL or DART) and lowering the price of an inherently simple device which should be accessible to everyone made me start this project.

For my concept two conditions need to be met:

  1. Designing a multispectral camera based upon a raspberry pi and a raspberry pi multiplexer, and writing the necessary software.
  2. Characterize the raspberry pi imaging sensor; extract the spectral response curves and set those free on the web.

1. TetraPi: a multispectral raspberry pi

Hardware

Designing the housing, either as a sled design to be mounted in a standard outdoor security camera housing and a fully independent camera has been fairly straightforward. Below you see the design of a camera sled which fits a VITEK security camera housing, as well as the stand alone / mobile version.

mobile

static

Both designs are largely finished and might need some cosmetic updates, but by and large these designs work. Features include a provision for four raspberry pi cameras (NOIR or RGB) and a filter holder which accomodates 1" dielectric filters as produced by for example Thorlabs. This modular design allows for simulateous acquisition of for example a photochemical reflectance index (PRI), a true normalized difference vegetation index (NDVI), as well as a standard RGB image. Multiplexing the cameras is taken care of by an IVMECH raspberry pi shield multiplexer.

envisioned mobile camera features
  • four channel camera [finished]
  • adaptable filters (1" dielectric filters) [finished]
  • 12-24VDC battery operated [ubec ordered]
  • local and remote (cable) trigger [3 mm jack connection in place, push button installed]
  • ad-hoc wifi access to upload images directly to a laptop [software issue]
envisioned static camera features
  • four channel camera [finished]
  • adaptable filters (1" dielectric filters) [finished]
  • 12-24VDC Power-over-Ethernet (PoE) or battery operated [ubec ordered]
  • UPS feature using [awaiting the pijuice shield]
  • ad-hoc wifi access to upload images directly to a laptop for remote sites [software issue]
  • wired and wireless time lapse image acquisition and uploads, for on the grid location [todo]

Operation of the mobile version is still dependent on a tripod to ensure a proper exposure. The mobile version will use a ubec to step down an external 12-24VDC source to 5V needed by the pi, alternatively it can be hooked up directly to the 5V output of for example a drone. The plywood design should make the whole setup not overly heavy. The current mobile version weighs in at < 250 gr, and there is room to cut back the size of the housing (probably ending up around 200 gr without a battery). There is also still room to shrink the design, but for convenience reasons I keep it larger as it makes it easier to work on the system.

CAD designs

The drawings of both cameras can be found here:

All designs are based upon 3 and 9 mm plywood or acrylic. For environmental reasons I try to minimize plastics as much as possible. The mobile version has no fixed power socket yet as I still have to decide on the size.

Budget
mobile version
  • Raspberry pi B+ : $30
  • 4 * raspberry pi camera : $120
  • 16GB micro SD card : $10
  • USB wifi adapter : $10
  • 5V ubec : $10
  • camera multiplexer : $96
  • real time clock : $10
  • small electronics : $4
  • plywood housing : $5
  • screws / stand offs : < $4

Total: ~$300

static version
  • Raspberry pi B+ : $30
  • pijuice battery pack UPS : $41
  • 4 * raspberry pi camera : $120
  • 16GB micro SD card : $10
  • USB wifi adapter : $10
  • 5V ubec : $10
  • camera multiplexer : $96
  • outdoor housing : $25
  • plywood housing : $5

Total: ~$350

Note:

I do not include the dielectric filters in the price as these are somewhat optional and run at $100 a piece. The prices listed above also indicate a full system, meaning 4 cameras where this might exceed the need of some people.

A complete system would run for less than $800 dollar (or 1/5 of a commercial system - in hardware cost). A NDVI system using one filter would set you back $400. Alternatively, a photochemical reflectance index (PRI) system (2 filters) will cost you a little south of $500.

Software

Little progress has been made on this part. I have some script which I can recycle from my PhenoPi project but I'm still waiting on the multiplexer and until it's arrival little can be done from a software development point of view.

2. Characterizing the OmniVision OV5647 imaging sensor

Inverse modelling of either the standard RGB signal or derived spectral features is based upon a thorough knowledge of the spectral response of a camera. The spectral response of a camera is defined as the wavelength dependent sensitivity of a given sensor channel (RGB).

For reasons unknown most imaging sensors do not come with this information. However, this spectral response can be measured using a monochromator (a rare piece of equipment) or if one knows the spectral response of your source light and the wavelength dependent characteristics of a diffraction grating.

Using both the a known well characterized light source (measured with a spectrometer) and a well characterized grating, it should be possible to back out the spectral response of the raspberry pi camera. A research note on this topic can be found here.

Blog post on this topic can be found on my personal website and will trickle through to the research notes as well.


I did this Help out by offering feedback!


People who did this (0)

None yet. Be the first to post one!


7 Comments

Nice. I'm trying to do the same thing with a beaglebone black. http://publiclab.org/notes/gpenzo/03-25-2015/ndvi-cape-for-the-beagle-bone Only I use a monochroom aptina. And less mega pixel. Will follow your progress. Nice to see another approach to the the same problem.

Reply to this comment...


Wow, that's an impressive project. I don't have the electronic engineering skills to finish such a polished product I fear. I use a more 'off the shelve' approach. It's nice to see you found a monochrome sensor that was fully characterized. It's probably easier if you order true components rather than finished products.

Any reason why you use a beaglebone instead of for example a raspberry pi? Although more powerful it also seems limited in terms of plug and play (due to the limited number of usb ports). I'm mentioning this because having wireless access seems like a huge advantage (the pi can be an ad hoc wifi point which could server as a server from which to pull data without plugging in). I'm working on such a system to download images for more remote setups. Not having to interfer with the camera setup is a huge bonus for permanent monitoring as well as the weather sealing of a setup. Getting data would be as simple as running up to a fixed camera, log in to it's local network, grabbing the images (and deleting the old) and moving on to the next. Similarly, the mobile camera could directly send the images to a mobile device / laptop when in the field. This is of course no use for drone mounted equipment but it might be a good idea to think outside the scope of the original project. Adding this functionality will probably not be hard but will be a huge advantage.

Would the filters be interchangeable? I don't see any filters in front of the lens so I assume they are in the assembly. I'm asking because there are other indices out there that might be of interest (PRI, and other narrow band indices).

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Hallo khufkens. Off the shelve approch is a very good way, if it does the job why not. I will keep an eye on your project.

The main reason for the beaglebone black is the real time requirements I have. I need to read and toggle pins very fast. And as far I know I cannot toggle pins or read them out with 20Mhz speed. The beaglebone black has two extra realtime processor running at 200mhz. My first version used one for each camera. My current version uses only one. Small advantage, beaglebone has a lipo charger in it. But when you run it from a lipo usb does not work. But in my case is oke will be hanging from a drone. The filters are indeed in the lens housing. One is mounted on the lens it self and the other on the cmos sensor. This board will be NDVI so I do not really bother about interchanging the filters. Next version will be a setup with 4 or 5 cameras. I know a farmer who is interested in 4 different indexes and will pay for the development :). I still need to make a list of indexes which are useful. And maybe make a board which can handle them all, 8 or 10 cameras. But for that I need to find a sponsor :)

One usb is enough for me. Can handle wifi or 3g usb modem when supply with 5 volt.

With the PI you do not have total control of the cameras, broadcom limit the amount of settings you can do by hardware.

I remember when I was looking for a camera, omnivison also had a monochrome camera OV7251. This one also has global shutter like the aptina.

Grayson.

Reply to this comment...


Hi, did you get the ov5647 spectral response?. I'm really interested in your project because I'm doing something similar but with an Aptina AR0130 that has a know spectral response. The aim of the project is create a open source multispectral camera for UAVs. You could see more details in: https://hackaday.io/project/6386-usensecam

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Hi khufkens Any news about your tetraPi? I will keep an eye on your project. Cheers Alex

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


I'm busy with work for the moment and have little time for my side projects. I'll probably pick it up again somewhere in autumn. I contacted some people at MIT to sort this out haven't heard back. I'll have to follow up on that. Will keep everyone posted.

Reply to this comment...


@juvinski awards a barnstar to khufkens for their awesome contribution!

Reply to this comment...


Login to comment.