Status of Ginga FRF-to-FITS Reformatting Project:

Phase IV Ginga processing is complete. The goal of this phase was to search the entire database for files which were not converted and to extract as much data from them as possible. Most of the science runs processed here had some type of telemetry error part way through. These files were processed partially so as to extract as much data as fsortac allowed. Others were not converted because fsortac did not handle such errors as gracefully as it could have. All of these files (8 total) were processed fully after modifying fsortac.

Phase III Ginga processing is complete. In this phase, very long PC-mode FRF files were processed. Because of the huge amount of computer memory required to process these files (465 of the total 25,385 Ginga mode switches), they had to be processed in pieces. Note that ALL of the data processed in Phase III were very long PC (timing) mode files. With this phase complete, there remain but a few files which need to be reprocessed or need special processing. 98% of the data is now on-line at the HEASARC!

Phase II Ginga processing is complete. In this phase of processing, files which required large amounts of computer memory for processing were handled. The project was delayed for three weeks due to a computer disk crash. Some of the Phase I data had to be reprocessed. 96% of the database is now complete.

Phase I of the FRF-to-FITS conversion is complete and on-line at the HEASARC! 93% of the database was processed during a two week period and without any special handling. Most of the remaining files require large amounts of memory to be processed directly. Phase II of the conversion will handle these files. Approximately 10% of the remaining files will need further processing, since they require such an enormous amount of computer memory to process. These files will be split into parts and processed in Phase III. It is expected that the conversion will be complete by January 1999.
Conversion of Ginga FRF files to FITS has begun. The bulk of the data will be released to the public by the end of October 1998. A small percentage of the data will need special processing, which will continue through November 1998.
The Ginga FITS Data System is being beta tested. It will be released by January 1999.

Version 1.0 of the Ginga FITS data system is nearing release. The sorting code, gngsort (a.k.a. sortac), has been converted to use FITSIO. "Bit level" testing of gngsort was done in April and May to ensure that output agreed with the original code. With this phase of testing complete, we are now ready to perform an end-to-end test. We plan to analyze science data with the help of Dave Smith, a Ginga scientist from Leicester. When we're sure that the FITS files contain all of the required information in the right format, we will begin converting FRF files to FITS format. We expect the data system to be released to Goddard some time in August.

A computer has been configured at Goddard to perform the conversion of Ginga FRF files to FITS format. fsortac has been recompiled and tested on this new platform, so the software and hardware are ready to go. The FITS files are currently undergoing a final review before the conversion is begun.

An I/O error excited by a bug Version 4 of DEC's f77 compiler has been fixed. The bug was noticed by scientists at Leicester. DEC's fix is to turn off all optimizations.

Complete conversion of the response matrix generation code.
Convert/test the 'gngsub' tool to subtract a background file from a source file.
Write a user's guide for the ginga FITS data system. Dave Smith (Leicester) had some useful comments.
Work on the Ginga project stopped until after ACIS calibration at XRCF. Back in the spring.

Convert/test the 'gngsim' tool.
Convert the ginga response matrix to FITS format. Verify that the file is correct, and write an accessor function.

Convert/test the 'gngfit' tool to fit an n-parameter linear curve to a background file. I had a lot of problems here because I use a different routine than Leicester uses. This will have to be tested in detail later.

Done testing 'fsortac' analysis mode cleaning.
Convert 'cleanall' (Leicester) to 'gngclean'. I discovered a bug in the code, and Dave Smith confirms the problem exists in the original software.
Test 'gngclean' on several Ginga FITS files.
Convert and test the 'gngsaa' tool.
A meeting was convened at GSFC to discuss our efforts with Julian Osborne of The University of Leicester. Julian Osbourne delivered the Ginga FRF archive to Steve Drake (HEASARC) on two 4mm tapes.

A meeting was convened at GSFC to discuss the current state of the project. The major thrust of the remaining contract is to convert and test the Ginga FITS Data System.

The Ginga FITS formats have been approved by the HEASARC FITS Working Group. Work begins on a data cleaning FTOOL.
Ginga FITS files are resubmitted to the HEASARC FITS Working Group. This should be the last round of submission to the group.
The next task is to redesign the algorithm which handles the allocation of dynamic memory to hold housekeeping data.
Ginga FITS files are resubmitted to the HEASARC FITS Working Group. This round of changes was rather insignificant.
Work has begun on an ftool to clean Ginga FITS files.

After significant changes to the FITS files based upon suggestions by Lorella Angelini of the HEASARC FITS Working Group, the Ginga FITS files have been resubmitted for a third round of approval.
The HDU which lists Ginga channels vs. array index number is now in a format which (we hope) will soon be acceppted by the FITS Working Group.
The SUBFRAME HDU has been completely reworked so as to include 4 sample wide FITS data columns for modes in which attitude was sampled four times per subframe. Other data modes have 1 sample wide FITS data columns.
Standard keywords were regrouped to make the files more readable.

Change the FITS format as suggested by the FITS committee. This required a major rework of the "SUBFRAME" and the "HK" extensions.
Resubmit Ginga FITS files to the FITS Committee.

Change software, recompile, and rerun 'fsortac' on several test FRF files. The first sub-frame of each science run now has counts; furthermore, data beyond the first sub-frame is intact.
Reconvert Dr. Negoro's test set, put the data on the Penn State Astronomy anonymous ftp site, and notify Dr. Negoro.
Verify that if this constraint were relaxed, then the code would run properly. 'fsortac' calculates the time for all data extracted from the frf (including Housekeeping data), so there should be no problem with accepting partial sub-frames.
All data taken at the sub-frame level is now written to Ginga FITS files in raw form. This data had been written only in accumulated form.
Submit example Ginga FITS files to the FITS Committee.
Dr. Negoro's assertion verified. Partial sub-frames were not allowed in the existing mode.
Dr. Hitoshi Negoro, a member of the ISAS staff, notes that the first sub-frame of data in each Ginga FITS file is missing.

Scott Koch releases version 1.0 of 'fsortac' to the Ginga team for verification. This release is a result of the fact that 'saextract' is now able to make standard light curves out of Ginga FITS files. 'xronos' can read the files and display them. Dr. Brian Elza played a crucial role in making this happen.

Penn State's High Energy Astrophysics group web pages are maintained by Scott Koch.
Last update: October 27, 1998