Abstract
Continuing the effort to define the specification limits of my Prototype PLab V3 Spectrometer (and then again for https://publiclab.org/notes/stoft/01-19-2016/otk-proto3), the issue of amplitude measurement stability had not yet been resolved. Having constructed my prototype from more rigid materials, instead of paper and Velcro, I was in a position to eliminate most of the mechanical variables; leaving only the light source and the camera. While I'm not able to independently, optically and accurately, measure the Solux halogen source light intensity, I was able to measure the lamp's current during a 15 min warm-up period and verify its very low drift. This observation, along with the collected spectral data over a 60 minute period suggests that, assuming mechanically rigid construction, the camera is the largest source of PLab spectrometer amplitude variability; easily adding 5-10% error.
References
PLab 3 Spectrometer Upgrade Prototype
Spectrometer DVD-Alignment Auto-Correction
Disclaimers
Since the development work to support automated data collection would have swamped the efforts of otherwise relatively simple observations like temperature and current, the sample interval was limited to 1 minute, for the first 15 minutes, and 5 minutes during the remaining 15-60 min test duration.
The "Raw UN-Corrected Data" label of the plots refers to the amplitude of the data NOT having been "gain-corrected" against a Solux 4700K emission curve. It does NOT refer to CFL wavelength calibration and the X-axis is labeled as "CFL-Corrected" where appropriate,
Solux Source
The light source for these observations is the broadband Solux 4700K halogen powered by an Anchorn halogen class-2 12VAC supply. Solux claims their 4700K halogen bulbs intensity remain stable within +/- 5% over their 4000 hour life. I interpret this to reference long-term drift but decided to make an independent measurement by monitoring the bulb's 12VAC current during the first 15 minutes after cold start from 22 degC room temperature. I also measured the lamp's surface temperature using a Kintrex, non-contact, laser thermometer. The following plot shows these two sets of data:
Note that to make it easy to show both curves, the measured current (~2.72 A) was simply scaled up by 30x. Also note that following a 1-2 minute warm-up period, the bulb's current consumption varied by <0.01 Amps -- less than 0.36 %. This is important because the light output is directly related to lamp current consumption. The stability, n this case, appears to be well less than 0.5% over time. This is strong evidence that the Solux 4700K lamp is a very stable source and any spectrometer spectral intensity variation over 1% should suggest spectrometer instability rather than source instability.
[Note: While the DVM observations were only recorded once per minute, the DVM can respond above 1 sample/sec -- I observed zero "flicker" in the readings during the 15 minute test period. This gives additional weight to the longer-term measure of lamp stability.]
Spectrometer
As noted above, I'm using my own Prototype V3 PLab Spectrometer, constructed from maple for mechanical rigidity. This assembly was secured to the same bench as the Solux source, at night, under minimal ambient room light. The source to spectrometer distance was held constant at 24 inches and thin-film attenuation was inserted directly in front of a 0.12mm film slit (provided with the PLab 'kit'). The 0.12mm slit provides sufficient accuracy to easily resolve the double-green peak of a CFL so will not add significant distortion to the detected Solux spectra.
To assure reliable spectral data acquisition, the PLab "thin-line" webcam data was extracted from USB via Matlab such that I could control the precise line of pixels providing the data to assure consistent measurements. Experiments showed that I had at least +/- 10 pixel lines to choose from with no discernible difference in data. This assured that line selection was not a variable.
Sampling
Two different time periods were of interest. First, there is a "warm-up" period for the lamp and second, longer-term stability was of interest. The simple solution was to observe temperature, current and spectrum on 1 minute intervals from '0' (cold start) to 15 min and then on 5 min intervals for the remainder of an hour. The choice of 15 minutes both convenient and allowed the lamp to achieve a stable operating temperature.
Spectra
While I have previously used the Solux lamp to research Intensity Calibration of the spectrometer, I did not use that correction of this spectral data because I wished to observe the entire detectable spectra and to observe the raw R/G/B mid-band signal data.
Following the validation of the Solux source stability, the spectrometer's configuration needed to validate the R/G/B channel data would never clip. The following plot shows that the attenuation was set to maximize the SNR while remaining linear.
Once the configuration was set and mechanically stable, the simplest first observation was to plot a selection of spectra acquired over time; using the same plot:
Clearly there is some form of variation, drift perhaps, and noise. So, to get a better visual image, all collected spectra were combined into a pseudo-3D "waterfall" plot just to visualize any dramatic trends; should they exist.
Analysis
The above plot shows no obvious dramatic trends but does suggest the primary instabilities are drift and noise. Since we have data for the three R/G/B data channels, I extracted a 10nm "mid-band" average for each channel and plotted that average over the lifetime of the measurements:
However, be aware of the illusion created by the reduced sample rate from 15-60 min. The variations "appear" to be "smoother" but that is not the case; there are simply only 1/5-th the number of data points. Look at the 1-15 min segment to better visualize the random drift and noise of the spectrum.
This lead to refining the data to extract the max / min limits and the max-min relative percentage error over both the short-term (0-15 min) and long term (15-60 min).
Note the following in the above two plots: 1) the first 15 min warm-up does add some variation to the max-min error across the full spectrum, 2) the relative percentage error behaves roughly like an intensity "offset" as the percentage rises as the signal drops and 3) at best, the long-term spectral intensity varies by at least 5-10%.
Note: This third point is important when comparing this value with the lamp current stability of <0.5% shown above.
To further isolate the 15 min lamp warm-up period from longer-term stability, R/G/B mid-band data from the (R+G+B)/3 derived spectra was extracted and then plotted along with the initial 15 min temperature profile. While the effect is small, there is slight more "variation" in the spectral intensity during the first 15 minutes of lamp operation. However, this should be expected.
Conclusions
There is evidence in the temperature and lamp current data which suggests the Solux 4700K lamp (with a 12VAC, halogen class-2 supply) can provide better than 0.5% long-term intensity output stability.
There is evidence that the Solux 4700K lamps achieve best steady-state performance stability after 15 min of continuous operation from a 22 deg-C room temperature startup.
This spectrometer V3 prototype is constructed of maple, was mounted to the same bench as the Solux lamp, no configuration changes were made during testing and data collection, the spectral band was sufficiently "wide" and did not drift during the tests and the same pixel line data was extracted for every spectra. Given the measured lamp stability and the spectrometer's configuration, the most likely component to cause the 5-10% spectral intensity variation of "drift and noise" is the webcam.
Given the relative lack of configurability of the simplistic webcam devices as diffraction sensors, the observations presented here suggest the camera is not providing fixed exposure control. This is potentially caused by an active AGC running near full gain because of the relatively dark field presented by the spectrometer's slit, diffraction grating and usual low-light sources which are limited to prevent R/G/B signal clipping. Electronic noise and camera "dark current" noise generally contain only higher frequencies and would not appear as dramatic as the longer-term variations observed during these tests.
9 Comments
Super note, Dave! It answers and raises questions:
This would be a great test to run with slight variations to learn other related unknowns. For example, I'd love to see (quantitatively) how much the added rigidity of your setup improves stability. Do you have an unmodified PL v3 spectrometer for comparison? As we discuss ways to increase rigidity in the design at a minimum of cost/complexity, I'm interested in how much added rigidity gains how much stability. But I know this may be outside the scope of your interest for this note!
Also -- this is a great test for folks to try to reproduce. We've been talking about a "reproduce-me" tag inviting others to try to re-run a test on their own, to increase replications. Maybe this is a good one to try that idea out with?
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Ah, I also want to connect this to yesterday's post by @viechdokter: https://publiclab.org/notes/viechdokter/04-13-2016/reproducibility-test-of-data-using-an-ir-lamp
Reply to this comment...
Log in to comment
Hi stoft, I am really impressed by your organised experiment design and statistics approach.Compared with your's my tests are kindergarten stuff... ;-)
When I did the reproducibility test with the CFL lamp I noticed it was 2 hours capturing spectra and 4 hours or more processing the data. I worked right into the wee hours. I imagine you must have worked 24/7 for this note, huh? Great work!
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
@viechdokter:, thanks; just a lot of years of practice. Not to worry, your's are logical and thoughtfully organized.
Jeff, thanks. Yes, new observations do tend to provoke new questions ;-)
1) Yes, because the "drift" / low-frequency noise is more predominant, some form of 'AGC loop noise' (might be digitally controlled) seems more likely to me. Other contributors, like shot noise / dark current noise, are more likely to be Gaussian and so, with a spectrum, appears as the smaller nm-scale 'hash' that commonly appears on the signal. Remember, the camera is expecting to be illuminated over its whole area so the camera's 'algorithm' (analog or digital) is working to that. Whereas the spectral band is just a narrow patch in an otherwise dark field; so the camera's response is likely 'full gain'. If the 'gain' were per-pixel, then the near-full-scale R/G/B of the spectrum should be quite stable -- but it doesn't appear to be.
2) The 12V source is regulated I believe (though not regulated like a DC supply. While there may be some high-frequency hash on the supply, the lamp acts like a low-pass filter (it's a thermal device so reacts slow) which is why I believed monitoring the current would be usable. The digital voltmeter can easily respond to at least 1 Hz and while I plotted only at 1 sample per minute, the meter readings remained solid; no flicker. So, I have no evidence the lamp or the supply is less stable than the 0.36 % I measured. A 120VAC lamp would be subject to line-voltage variations; which are generally small and generally slow. If your lights are not flickering, you might see 1-2 VAC shift over a minute (if line voltage is drifting at all) which would only be 1-2 % worst case. The larger difference for 120VAC might be the intensity difference from one day or week to the next.
3) Yes, I noted the macro for SWB. Unfortunately, the issues are more basic than that. Often, the latency gets excessive which hangs scripts which affect not only extracting data and plotting but recovery of data -- which then must be re-extracted for analysis. It's much easier, and much more reliable, for me to extract, store, process and plot all with Matlab and all locally without the web and scripts overhead.
4) Quantifying stability, relative to construction, would be difficult (shaker table?) and impossible relative to potential user effects on a paper and Velcro assembly. I still have the paper version (though it did start to fall apart rather quickly) just do not know how to measure and quantify those effects on measurement stability. With the V3 kit, when I touched the cable or the paper box, tried to hold it still with both hands or even just let it sit on the desk, the recovered intensity profile would visibly shift, jump or drift. This is NOT the characteristics of a stable measurement device and I don't need to figure out how to quantify that drift vs type of handling.
The big clue is that optical benches are made of granite slabs not paper and Velcro. I simplified and used maple and glue. This is a typical failure of the minimal-cost strategy of product development where allowing performance to be just "pretty good" in order to save 5-10% of the product cost is, like the parallel in science, just bad engineering which will come back to bite you in the end. A scientific device, of any grade, absolutely must be constructed to meet viable specs. A paper and Velcro design for a stable optical device with better than 5% stability and <=2nm resolution is simply not going to be possible.
So, yes it is good to have others looking at the reproducibility of their results -- even doing so on a routine basis as part of the scientific process. Always be your own skeptic. However, if PLab's goal is to provide a stable and well specified spectrometer for many users to make their own reliable observations, the design must be substantially improved over the existing product. This research note simply shows the present limits to intensity measurement stability.
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Thanks, Dave, very helpful answers. I understand what you're saying about the difficulty of structuring a measurement of structural stability -- I'm just trying to think how we'd measure the comparative rigidity (and gained stability) of a design that's more manufacturable than maple (though as a woodworker I do love it as a building material :-) ). Say, if someone put up a laser cut acrylic design (I have some ideas in that direction), how would we assess if it's sufficient? Thanks!
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
Jeff; I'm glad I was clear. The mechanics is like the rest; it starts with the objective / spec that it is an optical device. That immediately rules out paper and Velcro where the optical components are not rigidly connected, but instead, were mechanically isolated from each other. Once the slit, diffraction grating and sensor are "tied together", then can you ask 'how well'. I would argue that that assembly should be free of internal shift with casual handling: 1) playing with the USB cable should have no effect on the signal, 2) zero flexure for up to at least 1 lb of pressure, 3) signal stability, while hand-held, should be as good as the handler is stable.
As to how to measure this, I'd suggest a real-time display of say a green peak from a stable lamp where the display is a live 3-5 point running average of that peak's value, plotted not to show the absolute value, but the difference from the 'norm' so as to observe drift. The analogy is an oscilloscope. Then, handle the device and watch for correlations between the mechanical and the signal change -- which should be rock solid relative to the mechanical mounting of the optical components and the USB cable.
As for materials, yes maple is a nice wood but I wouldn't rule out wood as a material as it can be inexpensive to cut in volume, provides rigidity when glued, can easily be light-tight and could be easy to ship a pile of pieces to assemble where the mechanical tolerances are good enough to assure accurate assembly. As for acrylic, when thick enough it is rigid but it does require at least laser cutting for accuracy and even the black isn't really light-tight by itself (based on my initial black acrylic housing I assembled as a replacement proto for the first PLab spectrometer), and the typically shiny surfaces are also very reflective so must be the covered with flat black paper, etc. This is not to say which material is best, at this point, but it does highlight the need for many 'thought experiments' to consider all the angles.
Reply to this comment...
Log in to comment
For anyone who is interested:
https://publiclab.org/notes/viechdokter/04-15-2016/thinking-about-a-more-stable-spectrometer
Reply to this comment...
Log in to comment
BTW I wonder if we could find out more about the effects of the camera if you could do this same setup plus statistics again let's say with a green or red laser pointer. The wavelengths will stay the same but the changes in amplitude might then only come from the camera electronics...? (if the laser produces a steady beam, that is...)
Is this a question? Click here to post it to the Questions page.
Reply to this comment...
Log in to comment
@viechdokter yes, having a stable source helps isolate the noise source. I chose the Solux as I could measure the current as an independent check of light stability -- but still not a first-order measure like a calibrated optical detector. However, the advantage is it is broadband so any wavelength shift has near zero effect. True, a laser or LED is frequency stable, but the simple pocket lasers are not guaranteed stable output (simple battery + voltage multipliers which are not well regulated) though I could design an LED supply that is very stable. However, the peak is very narrow so measuring the same camera pixel line for detection becomes a possible source of drift error as the effective "gain" of being on either side of a sharp peak only ads noise and during the measurement it is difficult to observe, and eliminate, such shift. Not impossible, but more complicated.
Reply to this comment...
Log in to comment
Login to comment.