Public Lab Research note


Using ICA for spectral analysis

by nathanathan | November 19, 2013 10:03 19 Nov 10:03 | #9809 | #9809

I've read a bit about some statistical analysis tools (in particular ICA and PCA) that seem like they could be applied to spectral analysis. The goal would be to devise an algorithm that returns the the relative emission/absorption of every known element and chemical compound in a spectrum.

Mathematically, I think this is something like finding solutions to c * A = m where A is a matrix where each row is a known spectrum, c is the vector we want to determine that contains the relative concentration of each spectrum, and m is the values of the measured spectrum as a vector.

This requires a database that contains the isolated spectrum of every element and chemical compound. It seems like by combining these spectra it should be possible to determine the chemical composition. However, I'm not sure if such a thing exists. Perhaps there is a way to filter or process the data from spectral workbench to find isolated spectra for individual chemicals.

Another problem is that there may be different chemicals or combinations of chemicals that have the same spectra. (Are there individual chemicals that have matching spectra?) Perhaps this could be surmounted by offering users a way to rule out solutions by choosing between different compositions.

I'm looking for links to similar work (I'm betting someone has thought of this before), and I'm also looking for comments on whether this seems horribly naive and what other issues I'm missing.

Update 1:

According to this wikipedia article different spectra may be needed to calculate the emission and absorption of an element. Emission can occur only at the absorption frequencies but the intensities are different.

There are multiple techniques for determining an element's emission and absorption spectrum. I'm a spectography newbie so I have some questions about them, such as:

  1. Can one use reflected light from an every day light source to determine emission and absorption? For example, is it possible to determine the chemical composition of, say, a key on my keyboard, just by pointing a spectrometer at it while a bright light is shining on it? Some information about the light source probably needs to be determined, perhaps this could be accomplished by taking a spectograph with a mirror (or maybe something that diffuses light more) placed in front of the keyboard. Furthermore, accommodating the light source may cause problems when doing ICA since all the components are dependent on the light source, but perhaps this can be overcome by multiplying all the reference spectra by the relative magnitude of the light-source spectrum.
  2. Does the experimental set-ups yield fundamentally different spectra? For example, does using a hotter flame or more intense radiation produce an emission spectrum without any emission frequencies?

My mathematical formulation is flawed because it doesn't factor in noise.

I posted my question "Are there individual chemicals that have matching spectra?" on the chemistry stack exchange. The answer is yes! Chiral molecules which are mirror images of each-other will have the same spectrum, although using polarized light it is possible to tell them apart.


10 Comments

Nathan, Would this procedure require a database of known spectra which included the relative (or absolute) strength of each emission peak or just the wavelength of each peak? I don't think anyone has calibrated a Public Lab spectrometer so that peak height can be compared in a very meaningful way. (The problem is that the cameras record data in three color channels which vary in sensitivity across their wavelength range and also overlap with each other.) If only the wavelength of each peak is required, this approach could be tested with fluorescent bulbs which have clear emission peaks from three elements (mercury, terbium, europium) with well known peaks in the visible and near IR range. Good databases of emission peaks exist, but it might be easier to make a test database by scraping the data associated with this image: https://en.wikipedia.org/wiki/File:Fluorescent_lighting_spectrum_peaks_labelled.svg

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Speaking of fluorescent peaks, I just tried your latest peak detection macro on a spectrum of a fluorescent lamp, then tried your new download macro and popped the peak trace into Excel, and Wham:

SnowyPeaks.PNG

That is the peaks (using all defaults) of my Snowy Sky CFL spectrum. It seems to be an excellent record of the spectrum. Very cool.

Reply to this comment...


@cfastie It is possible to use binary ICA if only the peaks are known. However, in binary ICA all smooth continuous spectra would look nearly the same, it throws away a lot of information so I would expect the results to be much less accurate. Calibrating the camera's image sensor so the the intensities are at least proportionally correct seems feasible, but I think it would require being able to photograph something with known intensities, which might not be easy to find. Maybe this could be accomplished by including standardized glow in the dark rainbow strips with the public lab spectrometers? Thanks for the fluorescent bulb suggestion. Are the emission peak databases you mentioned behind a pay-wall? And do they only include the peak locations? I'm glad you're happy with the macro :)

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


There are plenty of open sources of spectral data. Some are listed at this Public Lab wiki. I think the NIST database is a good standard with reasonable filtering and downloading capability. It has peak location and intensity, but includes lots of minor peaks that Public Lab spectrometers probably won't detect.

If you take a spectrum of the sky, you can see the inconsistent effect of the three color channels. Assuming that the intensity should be constant from 400 to 900 nm, you can take the non-constant spectrum and derive the function to correct it. That will be valid only for that camera at that white balance setting. But it is probably worth doing if you want to compare peak intensity.

Reply to this comment...


No reason to assume that intensity is constant from 400 to 900 nm, because we apparently know that it is not:
SunlightSpectrumGraph.jpg

So use a curve like this and correct a sky light spectum from your spectrometer to match it. Then use that correction on all spectra from that spectrometer. Make sure you keep white balance the same (no auto WB!) and don't over expose any peaks in any channels. I'm not really sure how to best take a spectrum of sunlight.

Reply to this comment...


I think bouncing a light off the surface of a substance is not going to produce atomic or molecular emission. You have to excite the atoms or molecules. Without that excitation, you mostly learn about the reflective properties of a surface and usually not much about its elemental makeup. Raman spectroscopy uses a laser to interact with the molecules in a substance and then detect shifts in some (scattered?) wavelengths. The Public Lab spectrometer is not there yet.

Some elements will produce emission lines when heated up a little (Na or Cu in a flame) and some will fluoresce at characteristic wavelengths when hit with higher energy light (a UV laser). An obstacle to detecting the signature wavelengths is that other emission in the gas flame or fluorescing substance is noise.

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


When calibrating against the sun it seems like the spectrum would vary significantly depending on atmospheric conditions (e.g. in a smoggy city, during a sunset or on a overcast day).

I think you've ruled out using emission spectroscopy for analyzing material composition with a simple experimental setup. But perhaps it's possible to use absorption spectroscopy to determine an objects composition from reflected light-bulb light.

Reply to this comment...


I guess you're right that taking a spectrum of the sun is not going to allow people in Santa Fe and Beijing to compare their peak heights.

Simple experimental setups that could allow elemental identification:

  • gas discharge tubes
  • flame spectroscopy (compare flame with flame+substance)
  • fluorescence (compare UV laser in medium with UV laser in medium+substance)
  • absorption spectroscopy (compare light transmission through liquid with liquid+substance)
  • gas absorption spectroscopy (compare light transmission through gas with gas+another gas)
  • others?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Hi @Cfastie, Do you have a reference for the sunlight spectrum you posted in your comment. for my research, I need a reliable reference for the complete spectrum of daylight. It should include the UVB, UVA, visible, and IR that exist in daylight. Any help?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


I found that image in Googleland, I'm not sure where. Wikipedia has a complete article with lots of references: http://en.wikipedia.org/wiki/Sunlight

Reply to this comment...


Login to comment.