Subsections

• 7.18.1 Actions
• 7.18.2 Usage and Inputs
• 7.18.3 Troubleshooting
• 7.18.4 Data Products

7.18 FIT_SPECTRA Stage

This stage is run at least once after MERGE_OBSERVATIONS has been completed to perform automated spectra fitting using the XSPEC spectral fitting package by Keith Arnaud. A suitable XSPEC script, meeting specific AE interface requirements described below, must be provided. Several such scripts, implementing simple XSPEC models, are distributed with AE (§5.12).

An excellent introduction to the concept of fitting X-ray spectra can be found in one of Keith Arnaud's PowerPoint presentations.

7.18.1 Actions

• The multi-ObsId source spectrum is grouped as described in §5.7.

• For each source AE prepends to the specified XSPEC script several TCL assignment statements that pass source information to the script:
  • set spectrum_filename "104357.47-593251.3.pi"
    The spectrum to be fit (ungrouped for the C-statistic; grouped for $\chi^2$).

  • set extra_spectrum_filename "104357.47-593251.3_grp5.0.pi"
    A second spectrum that should be plotted against the model. AE and AE-supplied fitting scripts use this mechanism to produce two plots for every source, a grouped spectrum (ldata.ps) and a cumulative ungrouped spectrum (icounts.ps), regardless of whether the fit statistic is $\chi^2$ or C-statistic.

  • set ignore_spec "1-34,549-**"
    The channels that should be ignored in spectrum_filename, as specified by CHANNEL_RANGE.

  • set extra_ignore_spec "1,98"
    The channels that should be ignored in extra_spectrum_filename, as specified by CHANNEL_RANGE.

  • set model_name "nogrp_tbabs_vapec"
    A name for the HDU that will contain the fit results in the file source.spectra.

  • set c_stat_flag 1
    A flag to specify whether the fitting statistic should be $\chi^2$ or the C-statistic.

  • set spectrum_description "NET_CNTS=150-10.7"
    A descriptive string for use in plot titles.

  • set bkg_ignore_spec "1-7,686-**"
    The channels that should be ignored in the background spectrum when fit with a model (CSTAT option).

  • set cplinear_energies {0.5 1.0 1.5 2.0 3.0 4.0 5.0 6.0 7.0 8.0}
    The energies at which the vertices of the cplinear background model (C-statistic only) are placed (§5.12).

  • set model_directory "/users/research/patb/TARA/code/ae/xspec_scripts"
    The path to the fitting components of the local AE package.

  • set interactive_flag 0
    A flag that is set when /INTERACTIVE is specified.

• XSPEC is then started in the source directory and is passed the modified script. The screen output of XSPEC is directed to the file xspec_run.log. Since XSPEC sometimes hangs during a computation, AE will kill the XSPEC process if it consumes 600 seconds of CPU time, print a warning message, and move on to the next source. Examination of the xspec_run.log for such sources often reveals that the hang occurred during the XSPEC error command. The AE-supplied fitting scripts allow one to skip the parameter error computation by setting the TCL variable skip_errors to one.

  When the $\chi^2$ statistic is used, AE will skip any source which has $<4$ groups ($<2$ ``noticed'' groups) in its spectrum. Most fitting scripts will of course require more groups than that to function; it's the responsibility of the script to check that the number of noticed groups is adequate before attempting the fit.

  Fitting scripts should assume that the multi-ObsId background spectrum, ARF, and RMF are listed in the standard FITS keywords BACKFILE, ANCRFILE and RESPFILE in the source spectrum.

• When XSPEC finishes, AE expects to find files named ldata.ps and xspec.log in the source directory. AE then runs a small L^ATEX program which formats ldata.ps and xspec.log together in a PostScript summary of the fit, named summary.ps.

• AE archives the fitting script and the files produced during the fit in a sub-directory spectral_models in the source directory.

• The fitting script is expected to have written the fit parameters it determined as FITS keywords in an HDU named using the script variable model_name (see above) in the file source.spectra. If the observer runs the FIT_SPECTRA stage multiple times, with different models and/or differently grouped spectra, then the file source.spectra should contain multiple HDUs each named for the spectrum+model used. If the source.spectra files are so created, then the COLLATED_FILENAME stage may be run to collate across the catalog the fit parameters for a given spectrum+model, producing a FITS binary table with columns holding the fit parameters, plus the sourcename, RA, and DEC. Any ``-'' characters appearing in fit parameter names are changed to ``_'' in the column names to keep IDL happy.

7.18.2 Usage and Inputs

acis_extract, srclist_filename, /FIT_SPECTRA, CHANNEL_RANGE=[value,value], MODEL_FILENAME=filepath, MODEL_CHANGES_FILENAME=filepath, /CSTAT, SNR_RANGE=[value,value], NUM_GROUPS_RANGE=[value,value], /GROUP_WITHOUT_BACKGROUND, /INTERACTIVE, FIT_TIMEOUT=value

EXAMPLE:
idl $\vert\&$ tee fit_spectra.log

acis_extract, 'all.srclist', /FIT_SPECTRA, CHANNEL_RANGE=[35,548], MODEL_FILENAME='~/my_models/tbabs_vapec.xcm'

EXAMPLE:
idl $\vert\&$ tee fit_spectra.log

acis_extract, 'all.srclist', /FIT_SPECTRA, CHANNEL_RANGE=[35,548], MODEL_FILENAME='~/my_models/tbabs_vapec.xcm', MODEL_CHANGES_FILENAME='~/my_models/NH_frozen.xcm', /CSTAT

EXAMPLE:
idl $\vert\&$ tee fit_spectra.log

acis_extract, 'all.srclist', /FIT_SPECTRA, CHANNEL_RANGE=[35,548], MODEL_FILENAME='~/my_models/tbabs_vapec.xcm', /CSTAT, SNR_RANGE=[1,5], NUM_GROUPS_RANGE=[6,250]

• The two-element integer vector CHANNEL_RANGE controls the range of PI channels (i.e. the spectral band) used for fitting. The values [35,548] correspond to the approximate energy range of 0.5--8 keV. When the spectrum is grouped (§5.7) these two values directly control the placement of the first and last grouping bin boundaries.

• The optional keywords SNR_RANGE and NUM_GROUPS_RANGE change the spectral grouping parameters. See §5.7 for the meaning of these parameters, each of which is a 2-element vector.

• If set the optional keyword GROUP_WITHOUT_BACKGROUND excludes the background from the computation of group significance: $SNR = \frac{SRC\_CNTS}{1 + \sqrt{SRC\_CNTS + 0.75}}$ This is provided for observers who prefer the traditional approach to grouping in which a threshold number of source counts is required for each group.

• MODEL_FILENAME should be the path to an appropriate XSPEC script (.xcm file).

• The optional MODEL_CHANGES_FILENAME parameter provides a ``hook'' for fitting scripts to provide a capability for the observer to modify the behavior of the script. The convention is that the fitting script should contain this line

  # AE CUSTOMIZATIONS
  

  after the script has set up the spectral model. (The scripts distributed with AE support this.) AE inserts after this line the contents of the file MODEL_CHANGES_FILENAME which should contain an XSPEC script fragment that changes the default behavior of the fitting script as desired. For example, the inserted script fragment could contain XSPEC commands to freeze a parameter, or to change an abundance.

  WARNING! It is very easy to introduce typograpical mistakes into the model-changes script that silently defeat the intended changes to fitting script variables that you intend, expecially since variable names in the TCL language are case-sensitive. We strongly recommend that model-changes scripts should verify that a variable name exists before trying to change its value. The example model-changes script below first makes a table of (name, value) pairs for the script variables you wish to update, then runs a little block of TCL code that verifies that the specified variable names already exist in the fitting script:

    set custom_value(kT0)     2.0
    set custom_value(NH0)     1.0
    
    foreach variable_name [array names custom_value] {
      if [info exists $variable_name] {
        set $variable_name $custom_value($variable_name)   
      } else {
        puts "ERROR: '$variable_name' is not the name of a variable in the fitting script!"
        tclexit 98
      }
    }
  

  To freeze a parameter, set its initial value and parameter range to the same value, for example:

    set custom_value(kT0_min) 15.0
    set custom_value(kT0)     15.0
    set custom_value(kT0_max) 15.0
    
    foreach variable_name [array names custom_value] {
      if [info exists $variable_name] {
        set $variable_name $custom_value($variable_name)   
      } else {
        puts "ERROR: '$variable_name' is not the name of a variable in the fitting script!"
        tclexit 98
      }
    }
  

  AE looks in two places for the specified MODEL_CHANGES_FILENAME. For example suppose
  MODEL_FILENAME='~/my_models/tbabs_vapec.xcm' and
  MODEL_CHANGES_FILENAME='~/my_models/NH_frozen.xcm'
  AE first looks for a file named tbabs_vapec_NH_frozen.xcm in the source directory. If that is not found then AE looks for a file at the exact path supplied, '~/my_models/NH_frozen.xcm'. In this way you can use a single XSPEC script fragment to modify all source fits in the same way, and/or you can modify individual source fits independently, e.g. specifying a different frozen $N_H$ value for each source.

  If an XSPEC script fragment is found its basename is appended to the model_name used in the various fitting files and in the HDU that stores the fit results, e.g. grp5.0_tbabs_vapec_NH_frozen.

  If you wish to apply multiple customization scripts then MODEL_CHANGES_FILENAME should be an array of file names.

• The optional keyword /CSTAT can be supplied to specify (via the c_stat_flag script variable) that XSPEC use the C-statistic. See §5.12 for a description the cplinear background which is used with the C-statistic.

  The AE package contains compiled versions of this model for several platforms. If the AE scripts cannot load this model you may need to compile it for your platform as shown below. Note the use of back-quotation marks around the uname shell commands.

  % cd <AE package>/xspec_scripts
  % xspec
  XSPEC12> initpackage  acis_extract  model.dat  ./
  XSPEC12> quit
  % setenv DIR `uname -s`_`uname -p`
  % mkdir $DIR
  % mv libacis_extract.* $DIR
  

• When the optional keyword /INTERACTIVE is supplied the fitting script will pause after the model is set up but before the source is fit. The observer can then interact with XSPEC to perform the fit (e.g. to adjusting abundances in a vapec model). When satisfied with the model, the observer should enter the command ``c'' (for continue); the fitting script will resume, calculating fluxes and producing the normal output data products.

  To use this feature, the observer must configure their XSPEC environment to access a small TCL program interact.tcl distributed with AE:

  % cd ~/.xspec
  % cp <AE package>/xspec_scripts/interact.tcl  .
  % xspec
  XSPEC12> auto_mkindex . 
  XSPEC12> exit
  

• The number of CPU seconds that XSPEC is allowed to consume can be changed via the FIT_TIMEOUT option.

AE's fitting scripts include a placeholder for loading a frozen observer-supplied model for background components that are not represented by the background observation that has been extracted. For example, when the ACIS ``stowed'' observations are extracted as the background spectrum for a diffuse source, then the observer will want to supply a model for the celestial background components (which are not present in the stowed data). You should ponder whether it is more appropriate for this celestial background model to be frozen or free:

• The placeholder in the fitting scripts shows how a background model, presumed to be frozen, is loaded and ``connected to'' the source spectrum you are fitting. This celestial background model would perhaps be derived by previously fitting a ``blank sky'' spectrum, extracted from a nearby patch of sky thought to lack emission from the diffuse object under study.

• If the ``blank sky'' spectrum you have available is very poor, then you might conclude that it is more appropriate to simultaneously fit the ``blank sky'' spectrum (connected to your celestial background model) and the source spectrum (connected to both your celestial background model and some source model). Modifications to AE's fitting scripts would be required for this approach, e.g. code to load the ``blank sky'' spectrum, code to connect it to the celestial background model, code to save the parameters of the celestial background model, etc.

7.18.3 Troubleshooting

If an XSPEC session encounters a problem it will often be able to notify AE that the fit was not completed successfully. AE will print an error message, and suggest that you look in the log file produced by the XSPEC run. That log file can be confusing to read, because the commands that are being run by the script are not (by default) echoed to the log file. Your best shot is to skip to the end of the log file and look for a message that gives a clue about the problem.

If the log file is no help, then your next debugging step should be to repeat the XSPEC run manually in a shell in order to reproduce the failure outside the context of AE. Although the unix command line that AE spawns to run XSPEC (shown in the IDL window) is complex at first glance, the heart of it is simply the normal syntax required to have XSPEC run a TCL script, for example:

    xspec  -  spectral_models/grp3.0_tbabs_pow.xcm

Although you can manually run the fitting script in that manner (supplying it as a parameter to XSPEC on the command line), you will often find that XSPEC will exit immediately after the problem is encountered, preventing you from exploring the state of the model, looking at the fit, etc. Thus, it is often better to start XSPEC and then load the script, for example:

    xspec
      @spectral_models/grp2.0_tbabs_pow.xcm

If the fitting script is running normally, but the spectral model parameters are running ``off into the weeds'', then you should first suspect that the problem is arising during execution of the ``error'' command. When a model parameter is poorly constrained, the ``error'' command can explore astrophysically extreme realms of parameter space while searching for a rise in the fit statistic commensurate with the specified uncertainty. Sometimes, a second and slightly better local maximum will be found during that search and will be adopted as the new best fit, even if its model parameters are astrophysically extreme (e.g. very high NH and very low kT in a thermal plasma). The best-fit XSPEC model that existed before estimation of parameter errors is saved in the file model_before_errors.xcm (see below). Estimation of parameter errors can be disabled by setting the skip_errors variable in the AE fitting script, a task most easily accomplished via the MODEL_CHANGES_FILENAME mechanism.

7.18.4 Data Products

Four data products will remain in the source directory:

{sourcename}_grp{significance}.pi:

the multi-ObsId source spectrum grouped with the indicated significance.

spectral_models/grp{significance}_{model_name}.xcm:

the script passed to XSPEC

spectral_models/grp{significance}_{model_name}:

a directory holding the following files which may have been written during the fitting session:

ldata.ps

A plot of the grouped spectrum with model.

icounts.ps

A plot of the cumulative ungrouped spectrum with model.

xspec.log

A text file showing the fit information.

summary.ps

Summary showing ldata.ps and xspec.log.

xspec_run.log

STDOUT from the XSPEC process.

model.xcm

The final best-fit XSPEC model that was written to the file source.spectra and was used to make plots.

model_before_errors.xcm

The best-fit XSPEC model that existed before the fitting script attempted estimation of parameter errors. This model will differ from that in model_before_errors.xcm in cases where the search for parameter errors revealed a better fit.

source.spectra:

a FITS file containing the fit results in an HDU named model_name.