Quantum Efficiency Uncertainties Due to Measurement Errors

To determine the accuracy of our model fits, given that the model is correct, we must define the volume of model parameter space within which model parameter values are consistent, at some level of confidence (we will adopt the 90% confidence level here,) with the data. This parameter space volume is bounded by a surface S₉₀, which we shall call the ``90% confidence surface''. To each point p on the confidence surface S₉₀, the model assigns a quantum efficiency vs. energy function f_p(E). The ensemble of such functions $\{f_{p}(E): p \in S_{90}\}$ is then the set of quantum efficiency functions just allowed, at 90% confidence, by the calibration data. A complete characterization of the quantum efficiency error would include a probability distribution of the allowed values of f_p(E) for all p in the parameter space volume bounded by S₉₀. We do not provide such a characterization here. Instead, we attempt in the following way to estimate the envelope of the functions f_p(E) allowed by the data.

First, we characterize the 90% confidence surface. This surface S₉₀ is defined by the property that for any point p on S₉₀, $\chi^{2}(p) - \chi^{2}_{min} = \Delta \chi^{2}_{crit}$, where $\chi^{2}(p)$ is the value of $\chi ^{2}$ computed for the model f(p) with respect to the data, $\chi^{2}_{min}$ pertains to the best fit model, and $\Delta \chi^{2}_{crit}$ is a constant depending on the number of model parameters constrained by the fit.

In chosing $\Delta \chi^{2}_{crit}$, we note that the four fitted parameters belong to two distinct sets. The three gate structure parameters are entirely constrained by the low-energy (E < 3 keV) data, while the depletion depth is entirely determined by the high-energy data (E=5.9 keV, in our case). For purposes of bounding the quantum efficiency errors, we can therefore consider the gate structure parameters separately from the depletion depth.

For three interesting parameters (the thicknesses of Si, SiO₂ and Si3N₄) at the 90% confidence level, $\Delta \chi^{2}_{crit} = 6.2$, the 90% confidence limit for a $\chi ^{2}$ distribution with 3 degrees of freedom. We have found the six points on S₉₀ which represent the maximum deviations (both positive and negative) of each of the three gate structure parameters from their best-fit values. These are listed in Table 4.64.
 

Table 4.64: Six points on the 90% confidence surface used to determine the uncertainty envelope for FI detector quantum efficiency.

 

Si	SiO2	Si3N4	Remarks
($\mu m$)
0.235	0.233	0.045	Minimum Si
0.306	0.229	0.016	Maximum Si
0.276	0.220	0.032	Minimum SiO2
0.264	0.243	0.029	Maximum SiO2
0.304	0.233	0.015	Minimum Si3N4
0.237	0.230	0.045	Maximum Si3N4

We have evaluated the quantum efficiency model at each of these six points, and then found the extreme values of predicted quantum efficiency (among these six models) at each energy. To determine the envelope at high energy, we have separately varied the depeletion depth through its (single-parameter) 90% confidence interval of $\pm 1.7 \mu m$. (Thus we have in fact sampled twelve points in parameter space, viz., the six points listed in table 4.64 at the two extreme values of depletion depth. This error envelope, expressed as a fraction of the best fit model quantum efficiency is plotted as a function of energy for the S2 detector in the lower panel of figure 4.91. The plot shows the absolute value of the 90% confidence limits; positive and negative limits have been calculated, but are nearly identical, and so only the positive limits are shown.
 
 

Figure 4.95: Comparison of XRCF Phase I relative quantum efficiency measurements to Model Predictions for front-illuminated detectors. Points are deviations between XRCF quantum efficiency measurements, relative to S2, and lines are estimated 90% confidence intervals for model predictions. Note that the models are based entirely on relative QE measurements made at MIT; the points shown were NOT used to constrain the models. The RMS of the residuals for energies 525 eV is 1.3%.

Two limitations of this error analysis must be made clear. First, we have only considered errors resulting from measurement of the flight device quantum efficiency relative to that of the reference detectors. In principle, precisely the same analysis can be performed for the absolute calibration of the reference detectors. In so doing we find that the parameter ranges allowed by fits to the reference detector response to undispersed synchrotron radiation are smaller than those shown in Table 4.64 by factors of three to six. Thus the measurement errors associated with the absolute calibration are in any case small. Moreover, since model parameter errors in the absolute and relative calibrations are presumably statistically independent, their effect on the error envelope is properly accounted for by adding a small term in quadrature to the envelope derived from the relative data alone. We have therefore chosen to ignore the effect of absolute measurement errors in the present analysis, and estimate that the resulting error in our error envelope is less than 10% of the error envelope.

A much more significant limitation of this error analysis is that we have only included effects of measurement errors on four of the seven model parameters. The three channel stop parameters have not been varied in either the absolute reference detector or relative flight detector measurements. While we have some understanding of the allowed confidence intervals for these parameters (see reference  [Pivovaroff et al.1998]), they are not constrained by measurements on the flight detectors. In principle a joint fit of the flight detector quantum efficiency data and sibling detector mesh experiment data could produce a statistically correct error estimate. We have not yet attempted to perform such a fit.

Neglect of uncertainties in the channel stop parameters may be a significant shortcoming in our error analysis. For example, we find that a 10% increase in the fixed value of each of the three channel stop parameters (the single-parameter 90% confidence limit allowed by the mesh experiments is about 10-15% in each parameter) produces a 0.5% change in the best-fit apparent synchrotron radiation intensity determined by a reference detector, averaged over the 0.4 to 4 keV band. This is comparable to the (spectrally averaged) quantum efficiency error plotted in Figure 4.91.

We believe that a mesh measurement of one of our absolutely calibrated reference detectors would provide us with quantitative information on the effects of channel stop uncertainties on the final model quantum efficiency errors.

As an independent check on these errors (and on the models!), we have compared XRCF measurements of quantum efficiency, relative to detector S2, to the ratio of model predictions for the appropriate detectors. Results are shown for seven FI devices in Figure 4.95. Also shown are the 90% confidence limits for a single model efficiency prediction. (The solid curve is NOT a predicted error envelope for the ratio of two model efficiencies.) The observed residuals are in good qualitative agreement with the quoted error limits, especially considering that the model is based entirely on fits to the MIT CSR data, while the data points come entirely from measurements at XRCF. It is clear that inclusion of XRCF data in model fits is appropriate, and would improve the accuracy of the models further, especially at energies below 0.4 keV.

Quantitative comparison suggests that the observed residuals exceed those expected from the model uncertainties and measurement errors alone, however. Excluding the three measurements at 277 eV, for which the estimated model errors are quite high (and for which the observed residuals are well within expected model errors), the mean residual for 54 measurements shown in figure 4.95 is -0.003, and the RMS deviation about that mean is 0.013. The root mean square residual (about zero, rather the mean) is also 0.013. Ignoring the correlation of model errors at different energies, and assuming that the model errors for different devices are independent, we expect the RMS deviation for the residuals, (accounting for the 0.006 1-sigma measurement errors) to be 0.0095. Thus the observed residuals are larger, by a factor 1.3, than those estimated from the model errors alone.

Several factors may account for this discrepancy. Since most of the variance in the observed residuals comes from 2 of the 54 points, (the S4 point at 705 eV and the S5 point at 8040 eV), it seems likely that some unknown systematic errors (at the 3-4% level) affect these points. With these fairly obvious outliers excluded, the residuals (at 0.009, RMS) are consistent with the model error estimates. In addition, one might expect the internal correlation of errors within any one model to increase the observed variance of the residuals. On the other hand, the model error envelopes do not account for uncertainties in the channel stop parameters, and so must be underestimates at some level.

In any event, we conclude that the our models predict relative quantum efficiency of the ACIS FI detectors with an accuracy better than 1.5%, RMS, over the 0.4 to 8 keV band. Our estimates of the errors in the models are reasonable agreeement with the data, though the errors might be understimated by 30% or so.