Image processing
From chi and h
|
There can be several reasons to process an image and not just leave it as it comes out of the camera. One way or another we want to suppress or remove things that we don't like and emphasise those that we do like. This is not just a question of taste. Bias, dark current and noise, objectively, do not belong in a good image. Removing or suppressing them is a good thing.
If the image as produced by the camera cannot be improved, we should not waste time trying. If the exposure is well balanced, not saturated, free of visible noise, without hot pixels or stripy bias patterns, and if it is already an 8-bit JPG with not too much nor too little compression, then we should leave the image as it is. But if we push our equipment to the limit, are looking for faint objects, fighting the bright city sky, or try to see small detail on another planet, then we usually need to process the images. Hence, we need to take images that can be processed. We want to avoid JPG compression and gamma correction, if possible. We rather have raw images with as many bits digitisation as possible. We take bias and dark frames, perhaps also flat fields. Finally, we take a number of images so we can stack them together into a better single one.
There is a little paradox in image processing that you should be aware of before taking images. In order that we can remove or suppress features like bias or noise from our images these features must be recorded in the first place. In particular, to reduce noise by stacking images, we must use an amplifier gain (ISO setting) and data format so that the noise is in the images. JPG compression is not very good at this, because it spoils the statistical independence of the pixels. Even in raw images a low gain could reduce the noise so far as to count less than one digitisation unit. That is not enough, the standard deviation of the noise must be more than one digitisation unit to record the noise properly. 1.5 digitisation units per standard deviation of the noise is perhaps an ideal value, enough to record the noise but not wasting precious digitisation bits on oversampling noise.
We can distinguish several phases of image processing, though we will often skip several of these:
- Data reduction. These are standard procedures to correct according to dark frames and flat fields, to suppress noise by stacking, and to subtract unwanted background. To carry out these tasks it is best to use a floating point FITS format, which supports the mathematical operations far better than an integer format with limited dynamic range an the potential of saturation.
- Quantitative analysis. Photometry and astrometry should be carried out on such reduced data, before further, more cosmetic processing.
- Cosmetic processing. Unless we throw away the images after quantitative analysis, we want them to look their best. In this phase, we will convert to a common graphics format like JPG or PNG, i.e. an 8-bit integer format capable of storing full colour.
- Visual analysis. An image is not always just a pretty picture. We might want to make statements about the content of our images, like how many spots the Sun has, whether an aurora was in progress, etc.
For processing, we need software. I use the Linux operating system, as most professional astronomers do. I have written my own suite of utilities [1] to stack frames and to apply dark and flat corrections. In addition I use the Starlink software collection [2,3,4,5,6]. For picture processing, I use The Gimp [7]. ImageMagick [8] is handy to display images, although it can also do processing. Most people will prefer the interactive processing of The Gimp, but ImageMagick's command-line approach can be very powerful at times. A vital role is played by dcraw [9] to convert raw data from vendor-specific formats to 16-bit PPM. The output from this is not as raw as I sometimes need, and so I have hacked the source code to avoid the colour balance processing that dcraw would insist on doing. A useful little utility is jhead [10] to list the exposure parameters from a JPG header.
Data reduction
What is noise?
Isn't that obvious? Noise is when my image shows pixel-to-pixel brightness variations where the imaged object - such as black sky - does not have such variations.
No, it is not that obvious. There are two things that look like that:
- Noise. This is a random contribution to the image brigthness. It does indeed vary from one pixel to the next. It also varies from one time interval to the next.
- Bias and dark current. This is a non-random contribution to the image brightness. It does not change from one time interval to the next. But it is a property of the individual pixel and varies from one pixel to the next.
Because in a single image they look the same, photographers and camera sellers may call both "noise". If we want to remove these unwanted contributions to the image, we need different methods for the noise and for what misnamed "noise".
If a camera offers built-in "noise correction" or "noise reduction", this may be simple bias/dark subtraction, or it may be an attempt in software to filter out noise and the pixel-to-pixel variations in the bias/dark contribution, or it may be both.
Bias, dark and flat
The bias and dark current contributions to the image are very easy to record. Simply take an exposure just like the proper image - same exposure time and gain setting - except you put the lid on the lens to leave the detector in darkness. Other things like the ambient temperature should also be the same, so best do this just before or after the proper images are taken. The resulting reference image contains the bias and the dark current. Simply subtract this dark frame from the real image to clean up the latter.
- I(x,y) = i(x,y) - d(x,y)
- I: corrected image
- i: uncorrected image
- d: dark (dark and bias) frame
A flat field is a reference image taken with the lid open, through the same optics as the real image, but looking at a uniformly bright object, ideally a white one. The exposure time will be different so that the flat field is not overexposed. A good method is to pull a white tee shirt tightly over the front of the lens. Take a dark frame matching the flat field exposures so as to correct the flat field for dark current and bias. The resulting flat field image is normalised to average value 1.0 and the real image is divided by it. The flat and the real image are each dark-corrected first, then the real image is divided by the flat field:
- F(x,y) = (f(x,y)-df(x,y))
- I(x,y) = (i(x,y)-d(x,y)) / F(x,y)
- f: uncorrected flat field
- F: corrected flat field
- df: dark frame matching the flat field
Observe that flat correction without prior dark correction of both the image and the flat field is not accurate.
The mathematical operations involved are not complicated. They can be carried out by my chicam utilities [1], with utilities from the Starlink KAPPA package [3], or more elaborately with the Starlink CCDPACK package [4].
Noise and stacking
Noise can also be recorded in a reference image. Simply take two dark frames and subtract them from each other. The dark current and bias are thus eliminated and what remains is the combined noise from the two dark frames. Such an image is of no use. It shows the random pattern of noise as it turned out when the two dark frames were taken. The real image has a different pattern of noise. That is the trouble with the random processes that cause the noise: Each time you look at it it looks different from last time.
We can turn this randomness to our advantage. If we take not one but four images of the same object, each contains the same signal from the object. Each contains a different pattern of noise. If we add up the four images, the noise partially cancels itself. Whereas the signal is four times as strong, the noise will only double. In general, if we take m images the signal s is multiplied by m, but the noise n is multiplied by √n. The signal-to-noise ratio increases by a factor √n:
- S = s m
- N = n √m
- S/N = (s/n) m/√m = (s/n) √m
- s: signal in single frame
- n: noise in single frame
- S: signal in stack
- N: noise in stack
In practice, stacking is not as simple as adding images. Chances are, the object has moved within the field of view between successive images. Stacking software will have to establish these frame shifts and undo them before adding the pixel values from all the frames into the stacked image.
My chicam software [1] uses a Fourier transform to determine shifts between frames, and this works well for any bright object on a dark background, provided the background surrounds the object along all image edges. If there is also a rotation of the field, then I use Starlink's CCDPACK [4] to (i) find the stars, (ii) identify them in all frames, (iii) determine the shift and rotation, (iv) resample each frame onto a common pixel grid, and (v) mosaic all frames together. If the lunar or solar limb runs through the frames, then neither chicam nor CCDPACK will work. As a last resort, one can use Gaia [6] to - tediously - locate one feature in each of them and thereby determine the frame shifts.
With the shifts known the full stacking process, including dark and flat correction looks like this:
- D = (1/m) ∑ d
- I = ∑ [(i-D)/F]
- d: individual dark frame
- i: individual stack frame
- D: master dark
- I: stacked image
This presupposes that you take as many dark frames as you take image frames for stacking. That is prudent to do. If you took only one dark frame and if all the frame shifts were zero, then the noise pattern of that single dark frame would imprint itself very strongly on the stacked image. The whole effort of stacking would be futile. If all frame shifts are non-zero, then in principle you can re-use the same single dark for each image frame.
Why stacking is a bad idea
Stacking sounds like a great way to reduce the noise in images. However, there is a much better way to reduce the noise: Take a longer exposure. Consider taking an exposure m times longer instead of taking m exposures and stacking them. The signal will still grow by a factor m, but the noise may actually not grow at all, certainly less than √m. To understand this you must know that much of the noise - if not most of it - is generated when the image is read out from the detector. In the extreme, one long exposure may have almost the same noise level as one short exposure.
Another thing to consider about stacking is how much effort and disc space goes into it. To double the signal-to-noise ratio, you need four frames. To double again, you need 16 frames. For another factor two, you need 64 frames. And so on.
Why stacking is necessary
Stacking is necessary when there is no alternative. In particular, longer exposure may not be an option:
- The camera might not take longer exposures. E.g., it might not have a "bulb" exposure setting. Also, the dark current might drowning the image. Or, the battery might fail before the exposure completes.
- The object might not permit a longer exposure. E.g., a bright star of interest might saturate before a faint star is detected. Also, the sky background might drown the image when exposed longer. Or, the object might set or the Sun rise before the exposure is complete.
- A longer exposure might blur the image. E.g., a planet at high magnification would bounce around the field of view due to scintillation or due to wind shaking the optics and mount. Or, we might have no tracking mount and stars would become trails in a longer exposure.
How to use stacking properly
When stacking, we have three parameters to play with to accomplish the best stacked image in the presence of noise. These are the exposure used for each frame, the gain or ISO setting, and the number of frames taken. These are the priorities for doing it right:
- Digitise the noise. The gain or ISO setting must be high enough that the standard deviation in the noise is digitised as at least 1.5 units. That provided, gain/ISO should be set as low as possible.
- Avoid saturation. The exposure time has to be short enough not to overexpose the brightest objects of interest.
- Maximise the signal to noise ratio. How?
- Maximise the exposure time of the frames. Without overexposing, of course.
- Maximise the number of frames. This is less efficient than extending the exposure of each frame.
- Increase gain/ISO. Chances are this is not possible, since we already maximised exposure time. But sometimes, exposure is limited by other factors like blurring and trailing.
Background subtraction
It is hard to find an observing site these days that does not suffer from some level of light pollution. In the UK, the bulk of the pollutant is orange due to the line spectrum of the sodium streetlights. Strictly speaking, this is a foreground and not a background, as the sodium scattering occurs closer to the camera than the emission of starlight. Since it spoils the dark background of the empty universe, we call it a background, anyway.
If you have applied flat-field correction and if the field of view is only a few degrees or less, then the background should be flat or at least have no more than a simple gradient. It is probably not possible to record the sky background separate from the object: Taking a picture somewhere else on the sky will in general have a different level of background and a different gradient.
The thing to do is to take the images with enough spare field of view where only the background - and inevitably a constant sprinkling of stars - is recorded. If the object of interest is quite bright, its area should be cut out of the image. The remainder can then be fitted with a flat or linear surface and that fit subtracted from the image proper. If no flat-field correction was made, the background will show vignette and a curved surface will have to be fitted.
For this kind of work, I use the Starlink KAPPA [3] package, which can "zap" an area in an image, fit polynomial or spline surfaces to the background, and carry out the necessary image arithmetic to subtract the fitted background from the original.
Quantitative analysis
With the image brought into a good standard state, we can carry out analysis like measuring and comparing the brightness of small sources, or measuring their positions. Starlink's Gaia utility [6] provides a convenient interactive interface to other packages that carry out these tasks.
It should be noted that this sort of analysis should use relatively raw data, corrected for bias, dark current and flat field effects, and with noise suppressed by stacking. The more subjective processing we will come to next should not be done before such quantitative analysis takes place.
Cosmetic improvement
Most astrophotographs are perhaps taken to have a nice picture to look at and to show to others, as much an object of art as a scientific record. Applying some of the improvements discussed here will make the image a better one. However, these methods can be driven to excess, making the result less pleasing to look at, and perhaps making them a dishonest record of what was being imaged. If you also want to carry out visual analysis of your images, then you need to be careful not to overdo the processing of this section.
The operations described here can be carried out easily with regular image processing software. On Linux, the standard utility is The Gimp [7], which is also available for other operating systems. chicam [1] has a few non-interactive utilities as well, which at the very least can be used to convert from the 32-bit floating point FITS format to the 8-bit full-colour formats more suitable in the present context.
Crop to smaller pixel size
Cropping the image to a smaller size can focus the viewer's attention on the object you want to show. It can also enable you to use the full image resolution. Say, a 1500x1000-pixel image is too large to put on a web page. If most of it is wasted on black sky anyway, then cropping to a few hundred pixels keeps the relevant information, and in fact makes it more visible.
Incidentally, chicam [1] has a crop utility that operates on FITS format.
Linear and nonlinear stretch
These operations change the brightness and contrast of the image. In a linear stretch, we move the black level up and the white level down. This makes the dark parts even darker and the bright parts even brighter, thereby improving contrast, and possibly also adjusting overall brightness. The tools to do this are interactive; trial and error is the best approach. This is called a linear stretch, because between the new black and white levels the new brightness values are a linear function of the old values: The contrast change is the same at all brightness values.
A nonlinear stretch adds more freedom (for abuse?) to the method. The interactive tool allows us to create a curved relationship between old and new brightness values. Often this is used to give an increased contrast at low brightness - boosting faint detail - while giving up some contrast at high brightness. In that case, the advantage over a linear stretch is that the bright parts are not pushed into saturation, but still retain some contrast. Conversely, when curved the other way, the nonlinear stretch can also selectively increase contrast at high brightness at the cost of less contrast at low brightness.
Perhaps more a case of abuse is, when the nonlinear stretch is used to create high contrast both at low and at high brightness. At medium brightness, the output then becomes a negative, also of high contrast. There can be applications for this sort of thing, but the common nonlinear stretch tool is perhaps not the tool to use, and a false-colour (or false-grey) process may be better.
chicam [1] has two utilities to convert FITS data into a common graphics format. One will combine three grey FITS images into a single colour one; this will apply a linear or logarithmic stretch. The other utility works on a single grey FITS image. It will apply two separate linear stretches: The first stretch - operating on the fainter parts of the input - gives a grey image; the second stretch - operating on the brighter parts of the input - gives a false-colour image. The first or second stretch can also be applied on their own rather than in combination with the other.
Unsharp mask
What the unsharp-masking process does, is to make a smoothed version of the image and then to subtract a fraction of that from the original. While small-scale structure is fully preserved, large-scale structure is partially suppressed. When done well, the result looks much crisper. Unsharp masking is not pure cosmetics. When imaging planets it is all but necessary to make visible the surface detail that has been recorded, but remains hidden in the limb darkening of the planetary disc.
There are at least two parameters to tweak in unsharp masking. One is the size of the smoothing function, the other the amount of smoothed image to subtract. How exactly these two parameters affect the outcome is a bit hard to predict. It is best if you play around and get a feel for unsharp masking.
chicam [1] also includes a utility to apply an unsharp mask, working on FITS images, however.
Rescale to different pixel size
Some images benefit from rescaling them down to a smaller size. This can make them appear crisper, such as hide the fact they are a bit out of focus. If there is no valuable information in the higher resolution, this also makes for a smaller file on disc and uses up less screen space, say in a web page. Sometimes the opposite is the case. An image of a planet has only 100 or 200 pixels, which may be only a tiny patch of colour on a big computer screen. Scaling it up can help. It may show detail that the eye loses looking at the small image. Of course, in reality, no further real detail is created in the image.
You can of course scale any pixel size to any other pixel size. However, if you want to reduce the effects from interpolating pixels, then you probably want to use only odd integer scaling factors when scaling up (i.e. 3, 5, 7, 9, etc.). Only with these factors will the new image contain all the original pixel values with new interpolated values added between. Choose other scaling factors and all new pixels are the results of interpolation of more than one old pixel.
When scaling down these factors are also best (1/3, 1/5, 1/7, etc.); even integers (1/2, 1/4, 1/6, etc.) are almost as good. With any integer scaling factor, the new pixels will be simple averages of several old pixels and no interpolation is necessary.
Visual analysis
Finally, the image is finished, and people will admire it on your website or when presented to your local astronomy club. But perhaps there is some information of interest in them as well, like how may spots the Sun had that day, or whether there was an aurora developing at the time. Such visual analysis is best carried out on a more or less final image, optimised for human viewing. This is different from quantitative analysis, which is better carried out on data after reduction but before cosmetic processing.
References
- Horst Meyerdierks (2007). "chicam 1.3.0". http://www.chiandh.me.uk/soft/.
- "Starlink Project". Wikipedia. http://en.wikipedia.org/wiki/Starlink_Project.
- Malcolm J. Currie, David S. Berry (2009). "KAPPA - Kernel Application Package". Starlink User Note, 95.36. http://starlink.jach.hawaii.edu/docs/sun95.htx/sun95.html.
- Peter W. Draper, Mark Taylor, Alasdair Allan (2006). "CCDPACK - CCD data reduction package". Starlink User Note, 139.17. http://starlink.jach.hawaii.edu/docs/sun139.htx/sun139.html.
- Malcolm J. Currie, G.J. Privett, A.J. Chipperfield, D.S.Berry, A.C. Davenhall (2008). "CONVERT - A format-conversion package". Starlink User Note, 55.24. http://starlink.jach.hawaii.edu/docs/sun55.htx/sun55.html.
- Peter W. Draper, Norman Gray, David S. Berry, Mark Taylor (2009). "GAIA - Graphical Astronomy and Image Analysis Tool". Starlink User Note, 214.37. http://starlink.jach.hawaii.edu/docs/sun214.htx/sun214.html.
- gimp - GNU Image Manipulation Program. http://www.gimp.org/.
- ImageMagick. http://www.imagemagick.org/.
- Dave Coffin (1997-2009). Decoding raw digital photos in Linux. http://www.cybercom.net/~dcoffin/dcraw/.
- Matthias Wandel (1999-2009). Exif Jpeg header manipulation tool. http://www.sentex.net/~mwandel/jhead/.
