Next:List of Major Modes Up:Flow of ASTRO-E Pileline Previous: Flow of ASTRO-E Pileline

Processing Flow

We summarize the flow of pipeline processing of ASTRO-E data. The processes from 0 to 1 run basically only at ISAS. First FITS File (=FFF) and attitude file, created at ISAS, is daily copied to GSFC via network, together with Raw Packet Telemety (RPT) file for backup. The processes from 2 to 9 paralelly run in ISAS and GSFC by the identical processing scripts.

0. Create RPT and attitude file

We basically create one RPT and attitude file for one "target" (=one sequent observation between two maneuvers), but divide it into multiple files when the observation length exceeds one day not to make the data size of one RPT too large (same as ASCA FRF).

The RPT is created in the following way.

1. Telemetry data are extracted from the SIRUIS database for a specified time region (start and end), and a temporary RPT is created. The time region is determined by referring exact maneuver start times in the command plan and should cover longer than actual pointing time. The temporary RPT contains both of data during the previous maneuver (maneuver to the current target) and those during the next maneuver.
2. Time assignment is made for the temporary RPT.
3. Attitude file is created for the temporary RPT.
4. Checking a telemtry status in the temporary RPT whether the satellite is during maneuver or not, we divide it into multiple files i.e., RPTs during pointing and RPTs during maneuver. We finally keep a RPT during the previous manauver and RPT(s) during the pointing to the current target. If the pointing data length exceeds one day, we devide them into more RPTs (one RPT per one day).
5. The attitude file is also devided in the same way as RPT. Attitude file and RPT corresponds one-to-one and should have the same file name except for extention, ".att" for attitude file and ".rpt" for RPT.

0'. Update orbit file

The orbit file is supplied by NASDA and is updated every week, which is mirrored to GSFC. We distribute it to PIs along with the FFF and Second FITS File (=SFF, or ``Calibrated FITS'') .

1. Create FFF and House-Keeping FITS files (=HK) from RPT (mk1stfits)

The "mk1stfits" programs produce FFF/HK(s), separately for each instrument and major mode. FFF, science event data, has the extension ".fff" and HK ".hk". Basically, this process runs only in ISAS; when it becomes necessary to reproduce FFF/HK from RPT by some reason, for example, that bugs are found in mk1stfits, the process runs also in GSFC.

2. Creates Extended HK (=EHK) files (astemkehk, other intstrument specific tools)

The EHK files have the same format as HK files and hence can be read by "makefilter". The ftool ``astemkehk'' produces an EHK file where attitude and orbit information are written. Other intrument-specific EHK files may be produced if necessary.

3. Create a filter file (makefilter)

The ftool "makefilter" produces filter files from the HK/EHK files. The filter files keep exact times recorded in the original HK files and the sequence of time columns becomes "or" of those for all the keyword. For the time when the data is not recorded in the HK file, interpolation will be required. Method of interpolation, determined by the instument teams, shall be specified by a configuration file.

4. Spatial calibration (aspect, xrscoord, xiscoord)

The ACT, DET, FOC, and SKY columns are filled in ``temporary'' SFF, which is basicaly a copy of FFF.

When XIS Window mode is used, "xiscoord" access the HK file to obtain the window HK information (start address and window width) for each event.

5. Energy calibration (xrsgainhist, xrspi, xrscrosstalk, xispi, hxdggainhist, hxdpi, hxdgrade)

PHA is convereted into PI. Several columns (such as grade for HXD) will be filled.

6. Split by minor modes (maketime, extractor)

The temporary SFF thus processed is splitted into more files by minor modes (such as window option of XIS), referring to the HK and/or the filter files. Also, necessary minor modes keywords will be put in the header, for example, width of the Window when XIS Window mode is used. For XRS, sorting of events by time may be necessary by ``fmensort''.

At this stage, creation of SFF has been completed. We call the SFF "unfiltered event file" which has the extention ".unf".

7. Screening (screeing script, maketime, gtisync, fmaskfilt, extractor)

Standard data selection (such as by earth elevation, cutoff ridgity) will be applied to SFF and cleaned event files (with the extension of ".evt") are produced. Data of non-observation mode will be eliminated (such as GCC mode of XRS) and excluded from the GTI. The event files will not be seperated for different bit-mode (TBD). XRS event files may be seperated for FILTER WHEEL used and for hi+med and low resolution (TBD). For the XIS, data at the calibration-source position are excluded by ``fmaskfilt''.

8. Make images, spectra, lightcurves (extractor, xrsimage)

9. Extract trend products (maketime, gtisync, fmaskfilt, extractor, instrument specific ftools)

"Trend products" are extracted from the SFF.