Correcting for Atmospheric Extinction
From the July 1992 issue of *International Comet Quarterly*,
Vol. 14, pages 55-59. Copyright 1992.
MAGNITUDE CORRECTIONS FOR ATMOSPHERIC EXTINCTION
by Daniel W. E. Green
Abstract. A standard procedure is outlined for correcting total visual
magnitude estimates of comets, when objects involved in making the estimates
are at low altitude (i.e., high zenith distances), to allow for the effects
of atmospheric extinction.
Among the numerous problems encountered by observers in making total
visual magnitude estimates of comets, perhaps the most complex significant
issue involves the many times that comets are observed at great zenith
distances in the sky, particularly below 20 deg altitude. Over the years,
there has been concern as to whether all observers contributing to the
International Comet Quarterly (ICQ) are correcting properly for this
complicated phenomenon, whereby celestial light passes through increasing
amounts of the earth's atmosphere (and is thus diminished) as the observer
views objects at decreasing altitudes above the horizon. For example,
stellar absorption is roughly 0.2 magnitudes per air mass at visual
wavelengths (Schaefer 1985); at 10 deg altitude, the air mass is nearly 6
(air mass is discussed below). Obviously, significant errors can be made
when observers do not correct for extinction, and of course improper
corrections can be worse than no correction at all.
In its regular tabulation of data, the ICQ includes notes that indicate
when an observer made an extinction correction for a particular observation
(denoted on the printed pages by an exclamation mark, !, between the date
column and the magnitude method column). In practice, visual observers
should correct for extinction whenever the comet and/or the comparison
star(s) is/are below altitude about 30 deg; above about 30-35 deg altitude,
the potential errors due to improper methodology, to comparison-star
magnitude problems, and/or to instrumental issues probably make the effort
at small extinction correction unnecessary. Eleven years ago, in the early
stages of organizing the ICQ archive of photometric data on comets, we
outlined some procedures for standardizing observations, and we mentioned
then that we would review the problem of differential atmospheric extinction
correction (Green and Morris 1981). This paper represents that long-overdue
review, and is intended to encourage observers to correct properly and
uniformly for atmospheric extinction when a comet and/or comparison star is
at low altitude.
Definitions and Equations.
Air mass (X) is the amount of air that one is looking through, and
viewing toward the zenith is looking through 1 air mass. The air mass, to a
rough approximation, is given by X = sec z = 1/(cos z), where z is the angular
distance from the zenith (in degrees), though this simple formula breaks down
quickly as one nears the horizon (cf. Henden and Kaitchuck 1982). In this
article, we refer to altitude (90 deg - z) as the angular distance of the
celestial object above the local horizon, and to elevation (h) as the
observer's physical height above sea level in kilometers. While air mass
actually differs for each type of extinction material in the atmosphere, a
good overall representation of air mass is given by Rozenberg's (1966) equation
X = [cos z + 0.025 e**(-11 cos z)]**-1
= 1/[cos z + 0.025 exp(-11 cos z)]. (1)
Note that at z = 90 deg, X = 40. The altitude of the object can be found by
the formula
sin(90 deg - z) = sin phi sin delta + cos phi cos delta cos Htheta, (1a)
where phi is the observer's latitude (positive/negative in the northern/
southern hemisphere), delta is the declination of the celestial object
(positive/negative in the northern/southern celestial hemisphere), and
Htheta is the local hour angle (measured westward from the meridian),
which is
Htheta = theta - alpha, (1b)
where theta is the local sidereal time and alpha is the object's right
ascension (Meeus 1982). The actual amount of atmosphere at a given air mass
will vary considerably from site to site, depending especially on observer's
elevation above sea level.
Hayes and Latham (1975; hereafter HL75) note that there are three sources
of extinction in the earth's atmosphere that must be considered when dealing
with ground-based astronomical photometry: molecular absorption, Rayleigh
scattering by molecules, and aerosol scattering. At wavelength lambda = 510
nm, which is the peak spectral response for the rods of the human eye used
in night vision (e.g., Bowen 1984), molecular absorption (which occurs in
spectral lines and bands) is rather negligible (see the graph by Tueg et al.
1977), although for altitudes under 10 deg, ozone can cause extinction > 0.01
magnitude per air mass (HL75). We adopt Schaefer's (1992) value Aoz =
0.016 magnitudes per air mass for the small ozone component contributing to
atmospheric extinction.
Rayleigh scattering by air molecules can be represented by the
following equation (after HL75 for lambda = 510 nm = 0.51 micron):
ARay = 0.1451 exp (-h/7.996) magnitude per air mass. (2)
Aerosol scattering is due to particulates including dust, water droplets,
and manmade pollutants, and the extinction due to this is generally given by
the formula
Aaer = A0 lambda**[-alphao] exp(-h/H)
magnitude per air mass, (3)
where the scale height, H, is usually taken as 1.5 km (HL75; however, this may
vary by a factor of 2 on any given night) and lambda is the observed
wavelength (in microns). The quantity alphao varies from site to site;
Tueg et al. (1977) and HL75 find typical values near alphao = 0.9, but
we adopt alphao = 1.3 after Angstroem (1961) and Schaefer (1992).
Schaefer remarks that the variation in A0 "is rabid . . . because the
aerosol component varies greatly on all time scales". Volcanic aerosols, in
particular, are highly variable from site to site and year to year. For
reasons stated under "Procedure", below, I adopt A0 = 0.05 as an
average value. Thus, we will take the extinction due to aerosols for the
human eye as
Aaer = 0.120 exp(-h/1.5), (4)
so that for elevations near sea level, Aaer is about 0.12 magnitude per
air mass.
Procedure.
Ideally, one should use only comparison stars that are at the same
altitude as the observed comet, so that so-called differential extinction
correction will not be necessary. But in practice, this is only possible some
of the time, so we provide now a recommended procedure for applying corrections
due to differential extinction. Given the air masses of the observed comet and
comparison stars using equation (1), the observer must compute the extinctions
from equations (2) and (4). Then let
A' is equivalent to ARay + Aaer + Aoz. (5)
Schaefer (1987) says that A' is typically about 0.15 at a good observing site
such as Cerro Tololo in Chile (h = 2.22 km) and is about 0.30 for a site near
sea level in the eastern United States; this is in good agreement with
equation (5), as, for example, A' = 0.016 + 0.110 + 0.027 = 0.15 mag for h =
2.2 km. He also notes (Schaefer 1985) that for an average night on a
mountain or a good night at a dry sea-level site, A' = 0.20, while in a
humid climate, values of 0.25, 0.3, and 0.4 correspond to good, average, and
poor nights, respectively. Choosing a value A0 = 0.05 for equation
(3) would be realistic for these typical values of A', which is why it
was used to derive equation (4). The reason for choosing to adjust A0
(and thus, Aaer) is that ARay is well defined and Aoz has a
much smaller contribution, so the variable nature of Aaer contributes
much more greatly to significant variations in A'.
The total extinction at a given air mass is then
MA approximately equal to A'X magnitudes. (6)
One must compute the expected extinctions for the comet,
Mc = A'Xc, (7a)
and for the comparison star,
Mstar = A'Xstar. (7b)
Let the actual visual magnitude of the comparison star be mstar, obtained
from a source catalogue; then the observed magnitude of the star is
ma = mstar + Mstar. (8)
Likewise, the apparent total visual magnitude of the comet, m1, is the
sum of the comet's real magnitude (m'1) and Mc. If the comet is
judged to be equal in brightness to the comparison star (m1 = ma),
the corrected total visual magnitude of the comet is then
m'1 = ma - Mc = mstar + Mstar - Mc. (9)
If the comet is judged to be x magnitude brighter or fainter than the
comparison star, one must subtract or add, respectively, this amount x
from/to equation (9).
Use of the Tables.
Table Ia provides values of MA for each degree of altitude in the
sky (given in column 1 as zenith distance, z, in degrees), assuming the
average value A0 = 0.05 mentioned above. Columns 2-6 list the extinction
calculated from equations 1, 2, 4, 5, and 6 for sea level (h = 0) and for four
different elevations above sea level: h = 0.5, 1, 2, and 3 km. Let
widehat(M) = MA - M[z=0], where M[z=0] is the extinction at the
zenith (1 air mass). One can see that at z about 35 deg (altitude about 55
deg), the extinction of a celestial object with respect to its apparent
brightness at the zenith is near a tenth of a magnitude at sea level (i.e.,
widehat(M) about 0.1), while at mountain elevations the extinction doesn't
reach 0.1 mag until closer to z = 50 deg (altitude 40 deg). Because of the
uncertainty in the magnitude estimate due to methodology, comparison-star
source problems, etc., it probably isn't necessary to worry about making an
extinction correction until widehat(M) >/= 0.2 mag, which will be at z about
55 deg (altitude about 35 deg) near sea level and z about 65 deg (altitude
about 25 deg) for mountain sites.
Because extinction can vary significantly at a single site from night
to night (even from minute to minute!), two additional tables are included,
which observers can use for more dry, winter-like conditions (Table Ib) or for
more humid, summer-like conditions (Table Ic). Table Ib was computed using
A0 = 0.035, and Table Ic using A0 = 0.065.
One can then use Table Ia, Ib, or Ic to make the correction, keeping in
mind that a differential extinction correction is what is necessary in
comparing the brightness of a comet to that of a standard comparison star.
As an example, suppose that an individual at sea level observes a comet at
10 deg altitude (z = 80 deg) to be slightly fainter than Star 1, which is at
altitude 13 deg, and more noticeably brighter than Star 2, which is at
altitude 7 deg. From a catalogue, it is found that the V magnitudes of Stars
1 and 2 are 7.0 and 6.6, respectively. Then add the calculated extinction
(from Table Ia) to each star's real magnitude, as in equation (8), giving,
say, ma1 for Star 1 and ma2 for Star 2, which would now be ma1
= 8.2 and ma2 = 8.8. One can then compare the comet's apparent m1
with ma1 and ma2, and given the above description for this example,
one would conclude that m1 = 8.4 (which now becomes ma in equation
9). Now using equation (9), and finding Mc = 1.6 from Table Ia, we find
m'1 = 8.4 - 1.6 = 6.8, which is the value to report.
Of course, when dealing with differential extinction, it is especially
important to use several (say, N) comparison stars and take an average value,
ma'1 = Sigma[m'1]/N. This is the value to report for
publication in the ICQ, placing a '!' before the magnitude on the report
forms, in the same column as the magnitude (or in column 26 if sending data in
archival machine-readable form); note, however, the new note 'flags' or
abbreviations introduced in the next paragraph.
Closing Remarks.
It is difficult to present a simple way for applying extinction
corrections for observations made at any site on Earth, because many complex
issues are involved, including some in addition to those mentioned above. For
example, a blue star will appear redder more quickly with increasing zenith
distance than will a red star (cf. Hardie 1962; Henden and Kaitchuck 1982).
In general, because the V photoelectric bandpass commonly used for
comparison-star magnitudes is redward of the peak spectral response of the
human eye's rods, observers should opt for comparison stars with color index
-0.2 < B-V < +0.7 (cf. Green and Morris 1982). Refraction not only affects
the degree to which red light and blue light are passed to the observer from
celestial sources, but also increasingly affects the actual air mass for
increasing z (Henden and Kaitchuck 1982). The recent contribution of
emissions from the Pinatubo volcano greatly increased atmospheric extinction,
at the rate of about 0.1 magnitude per air mass (cf. Grothues and Gochermann
1992), to the extent that the Tables of this paper would not be indicative of
the tremendous brightness loss of celestial objects during the second half of
1991.
Because the problems are so complex within about 10 deg of the horizon,
we are implementing two new note codes, which we ask all observers to use when
the comet and/or comparison star(s) is/are at or below 10 deg altitude: '$'
is to be used instead of '!' when extinction corrections as outlined in this
paper are applied to objects at such low altitudes, and '&' is to be used when
the comet is < 20 deg and no extinction corrections are applied. When one of
the three tables of this paper is used in making the extinction correction,
especially above 10 deg altitude, we encourage observers to use the letters
'a', 'w', and 's', corresponding to Tables Ia, Ib, and Ic, respectively. If
one or more of the celestial objects involved in making the magnitude estimate
is below 10 deg altitude, use of the symbol '$' is suggested over the letter
codes. Effective immediately, the symbol '!' should only be used when
extinction corrections different from that described in this paper are applied
to objects at altitudes > 10 deg (z < 80 deg). [On the regular report forms,
extinction note codes are to be placed before the magnitude estimate, in the
same column as the magnitude estimate. When sending observations in computer
form, place this note in the column immediately preceding the observer code;
see instructions below.]
It is worth repeating here: when a comet is under 10 deg altitude, the
observer should first try locating comparison stars at the same altitude as
the comet, to avoid making an extinction correction. If there are simply no
such stars available, make every attempt to use stars within 1-3 deg of the
comet in altitude, in order to increase the accuracy of the extinction
correction.
Observers also must accurately determine z (or, alternately, the
celestial objects' altitudes) in order to use any extinction correction
properly, being careful to note that the horizon is probably not at 0 deg
altitude but rather some amount higher; one simple device to determine
altitude (without performing a calculation from spherical trig formulae
such as equation 1a above) is a protractor with a weighted plumline down the
center. Despite all of the issues involved, careful use of the procedure
described in this article should give reasonable magnitude corrections due to
the atmospheric extinction that are good to 0.1-0.3 magnitude, within the
"noise" of m1-estimate scatter caused by other problems such as
improper methodology and poor comparison-star magnitudes.
Acknowledgements. The content of this paper was greatly improved by
comments from Brad Schaefer (Goddard Space Flight Center), Arne Henden (Ohio
State University), Douglas S. Hall (Vanderbilt University), Brian G. Marsden
(Harvard-Smithsonian Center for Astrophysics), and ICQ Associate Editor
Charles S. Morris, who all critically read the manuscript in various stages
prior to publication. I also thank Brad Peterson (Ohio State University)
for directing me to some useful resources, and G. Antonio Milani (Istituto
di Patologia Generale, Padova, Italy) for sharing some of his extinction
calculations with me.
REFERENCES
Angstroem, A. (1961). Tellus 13, 214.
Bowen, K. P. (1984). Sky Tel. 67, 321.
Green, D. W. E.; and C. S. Morris (1981). ICQ 3, 67.
Green, D. W. E.; and C. S. Morris (1982). ICQ 4, 5.
Grothues, H.-G.; and J. Gochermann (1992). The ESO Messenger, No. 68, p. 43.
Hardie, R. H. (1962). in Astronomical Techniques, ed. by W. A. Hiltner
(University of Chicago Press), pp. 184ff.
Hayes, D. S.; and D. W. Latham (1975). Ap.J. 197, 593. [abbreviated HL75]
Henden, A. A.; and R. H. Kaitchuck (1982). Astronomical Photometry (New York:
Van Nostrand Reinhold), pp. 86-87; 106-107.
Meeus, J. (1982). Astronomical Formulae for Calculators, second edition
(Richmond, VA: Willmann-Bell), pp. 43ff.
Rozenberg, G. V. (1966). Twilight: A Study in Atmospheric Optics (New York:
Plenum Press), translated from the Russian by R. B. Rodman, p. 160.
Schaefer, B. E. (1985). Sky Tel. 70, 262.
Schaefer, B. E. (1987). Sky Tel. 73, 426.
Schaefer, B. E. (1992). Personal communication.
Tueg, H.; N. M. White; and G. W. Lockwood (1977). A.Ap. 61, 679.
Table Ia. "Average" Atmospheric Extinction in Magnitudes for
Various Elevations Above Sea Level (h, in km)
z h = 0 h = 0.5 h = 1 h = 2 h = 3
1 0.28 0.24 0.21 0.16 0.13
10 0.29 0.24 0.21 0.16 0.13
20 0.30 0.25 0.22 0.17 0.14
30 0.32 0.28 0.24 0.19 0.15
40 0.37 0.31 0.27 0.21 0.17
45 0.40 0.34 0.29 0.23 0.19
50 0.44 0.37 0.32 0.25 0.21
55 0.49 0.42 0.36 0.28 0.23
60 0.56 0.48 0.41 0.32 0.26
62 0.60 0.51 0.44 0.34 0.28
64 0.64 0.54 0.47 0.37 0.30
66 0.69 0.59 0.51 0.39 0.32
68 0.75 0.64 0.55 0.43 0.35
70 0.82 0.70 0.60 0.47 0.39
71 0.86 0.73 0.63 0.49 0.40
72 0.91 0.77 0.66 0.52 0.43
73 0.96 0.81 0.70 0.55 0.45
74 1.02 0.86 0.74 0.58 0.48
75 1.08 0.92 0.79 0.62 0.51
76 1.15 0.98 0.84 0.66 0.54
77 1.24 1.05 0.91 0.71 0.58
78 1.34 1.13 0.98 0.76 0.63
79 1.45 1.23 1.06 0.83 0.68
80 1.59 1.34 1.16 0.91 0.74
81 1.75 1.48 1.28 1.00 0.82
82 1.94 1.65 1.42 1.11 0.91
83 2.19 1.86 1.60 1.25 1.03
84 2.50 2.12 1.83 1.43 1.17
85 2.91 2.46 2.13 1.66 1.36
86 3.45 2.93 2.53 1.97 1.62
87 4.23 3.59 3.10 2.42 1.99
88 5.41 4.59 3.96 3.09 2.54
89 7.38 6.26 5.40 4.22 3.46
90 11.24 9.53 8.23 6.42 5.28
Table Ib. "Winter" Atmospheric Extinction in Magnitudes for
Various Elevations Above Sea Level (h, in km)
z h = 0 h = 0.5 h = 1 h = 2 h = 3
1 0.25 0.21 0.19 0.15 0.13
10 0.25 0.22 0.19 0.15 0.13
20 0.26 0.23 0.20 0.16 0.14
30 0.28 0.25 0.22 0.17 0.15
40 0.32 0.28 0.24 0.20 0.17
45 0.35 0.30 0.26 0.21 0.18
50 0.38 0.33 0.29 0.24 0.20
55 0.43 0.37 0.33 0.26 0.22
60 0.49 0.42 0.37 0.30 0.25
62 0.52 0.45 0.40 0.32 0.27
64 0.56 0.48 0.43 0.34 0.29
66 0.60 0.52 0.46 0.37 0.31
68 0.65 0.57 0.50 0.40 0.34
70 0.72 0.62 0.55 0.44 0.37
71 0.75 0.65 0.57 0.46 0.39
72 0.79 0.69 0.60 0.49 0.41
73 0.84 0.72 0.64 0.52 0.43
74 0.89 0.77 0.68 0.55 0.46
75 0.94 0.82 0.72 0.58 0.49
76 1.01 0.87 0.77 0.62 0.52
77 1.08 0.94 0.82 0.67 0.56
78 1.16 1.01 0.89 0.72 0.60
79 1.26 1.10 0.97 0.78 0.66
80 1.38 1.20 1.06 0.85 0.72
81 1.52 1.32 1.16 0.94 0.79
82 1.70 1.47 1.29 1.05 0.88
83 1.91 1.65 1.46 1.18 0.99
84 2.18 1.89 1.66 1.34 1.13
85 2.53 2.20 1.93 1.56 1.31
86 3.01 2.61 2.30 1.86 1.56
87 3.69 3.20 2.82 2.28 1.91
88 4.72 4.09 3.60 2.91 2.45
89 6.44 5.58 4.91 3.97 3.34
90 9.80 8.50 7.49 6.05 5.08
Table Ic. "Summer" Atmospheric Extinction in Magnitudes for
Various Elevations Above Sea Level (h, in km)
z h = 0 h = 0.5 h = 1 h = 2 h = 3
1 0.32 0.26 0.22 0.17 0.14
10 0.32 0.27 0.23 0.17 0.14
20 0.34 0.28 0.24 0.18 0.15
30 0.37 0.30 0.26 0.20 0.16
40 0.41 0.34 0.29 0.22 0.18
45 0.45 0.37 0.32 0.24 0.19
50 0.49 0.41 0.35 0.26 0.21
55 0.55 0.46 0.39 0.30 0.24
60 0.63 0.53 0.45 0.34 0.27
62 0.68 0.56 0.48 0.36 0.29
64 0.72 0.60 0.51 0.39 0.31
66 0.78 0.65 0.55 0.42 0.34
68 0.85 0.70 0.60 0.45 0.36
70 0.93 0.77 0.65 0.50 0.40
71 0.97 0.81 0.69 0.52 0.42
72 1.02 0.85 0.72 0.55 0.44
73 1.08 0.90 0.76 0.58 0.47
74 1.15 0.95 0.81 0.61 0.49
75 1.22 1.01 0.86 0.65 0.53
76 1.30 1.08 0.92 0.70 0.56
77 1.40 1.16 0.99 0.75 0.60
78 1.51 1.25 1.07 0.81 0.65
79 1.64 1.36 1.16 0.88 0.71
80 1.79 1.49 1.26 0.96 0.77
81 1.97 1.64 1.39 1.06 0.85
82 2.19 1.83 1.55 1.18 0.95
83 2.47 2.06 1.75 1.32 1.07
84 2.82 2.35 1.99 1.51 1.22
85 3.28 2.73 2.32 1.76 1.41
86 3.90 3.25 2.75 2.09 1.68
87 4.78 3.98 3.38 2.56 2.06
88 6.11 5.09 4.32 3.28 2.63
89 8.33 6.93 5.89 4.47 3.59
90 12.68 10.56 8.97 6.80 5.47
Astronomical Headlines