5.7 Grouping of Spectra

AE groups the multi-ObsId source spectrum using an algorithm which we believe is superior to that in the grouping tools in FTOOLS or CIAO in several ways.

• The placement of the first and last grouping bin boundaries are directly controlled by the observer through the AE keyword CHANNEL_RANGE. For example if CHANNEL_RANGE=[35,548] is specified, then the first group would cover PI=1:34 ($\sim<$0.5 keV) and the last group would cover PI=549:1024 ($\sim>$8 keV). By subsequently ``ignoring'' the first and last groups during the fitting process the observer is able to control the energy range of the fit to a resolution of 1 PI channel, rather than a resolution of 1 group (which can be large).

• Instead of defining groups so they contain at least a minimum number of counts in the source spectrum, AE defines groups so that in the background-subtracted (net) spectrum each group achieves at least a specified significance (signal to noise ratio). In situations where the background level is high, e.g. in spectra of diffuse objects, this approach can produce higher quality grouping. The error (noise) on the net counts in each group is computed from Gehrels' upper limits terms on the source counts and background counts in the group.

  Grouping always involves a trade off between the number of groups and the significance of the groups. AE's management of this trade off across a wide range of source counts is controlled by four parameters consisting of a range of acceptable group significance (keyword SNR_RANGE in §7.18) and a range on the number of groups that is acceptable (keyword NUM_GROUPS_RANGE). Suppose for example SNR_RANGE=[1,5] and NUM_GROUPS_RANGE=[6,250]. AE starts by constructing groups with a significance requirement of 5 ( $SNR\_RANGE_{max}$). If the resulting number of groups is between 6 and 250 then those groups are accepted.

  If instead (in this example) there are less than 6 ( $NUM\_GROUPS\_RANGE_{min}$) groups then AE iteratively lowers the significance requirement and regroups until there are more than 6 groups. If the significance requirement reaches 1 ( $SNR\_RANGE_{min}$) then whatever groups exist are accepted.

  If instead (in this example) the initial grouping produced more than 250 groups ( $NUM\_GROUPS\_RANGE_{max}$) then AE iteratively raises the significance requirement and regroups until there are less than 250 groups.

• The standard grouping algorithms are asymmetric. Each group is built from left to right (low energy to high energy). After the start of the group is defined, the algorithm adds channels to the right until enough counts are found to meet the specified criterion. All groups will end on a non-zero channel; when the spectrum contains a significant number of empty channels many groups will begin with a string of empty channels. This asymmetry can lead to very harmful artifacts in the grouped spectrum. For example, imagine this extreme example:
  • Groups are defined by requiring 5 counts in each group.
  • Virtually all the source counts are above 3 keV, i.e. the source is very hard.
  • Due to Poisson luck, or due to ACIS afterglows, or due to a soft background spectrum there are 5 counts within a few channels of each other down at 0.5 keV, and the algorithm allows those to form a very narrow first group.
  • The second group will then be very wide, encompassing the wasteland of zeros from 0.5 keV to 3 keV.
  The resulting grouped spectrum will arguably be a gross distortion of the truth: a high-flux narrow group followed by a low-flux wide channel. A $\chi^2$ fit will take these misleading flux measurements at face value, and the model may be significantly biased.

  AE's grouping algorithm attempts to mitigate this problem by selecting group boundaries which fall mid-way in the run of zeros (if any) following the group.