Subsections

• 5.1.1 ISOCAM use of time keys

5.1 Pipeline Processing

In common with the other instruments, every ISOCAM observation was subject to an automatic processing, known as the pipeline or OLP (Off-Line Processing). What was anticipated to be the final OLP v10 software was released in 2001 July. The pipeline ends with the ingestion in the ISO Data Archive (IDA) of products designed both for further analysis and to afford an initial assessment of the scientific value of the observational data. There are many different products involved. While the most immediately useful ones are probably those few that report images and point source measurements in physical units that are available for instant inspection in IDA (see Section 7.4), there is a variety of supporting material whose primary purpose is to provide a complete set of raw ingredients to enable the user's own analysis of whatever complexity. While most of the products are unique to ISOCAM, some hold auxiliary data, such as schedule, pointing and orbital information. This chapter describes the ISOCAM specific data products available in IDA and reviews the methods used in their derivation. Any caveats of which the user should be aware when interpreting these products can be found in Chapter 8.

ISOCAM data products result from one of the following tasks:

• automatic off-line data analysis of all observations;
• in-orbit calibration of dark currents, flat-fields, point spread functions, etc;
• ground calibrations;
• Quick_Look_Assessment of near real-time data from the satellite.

The products are written in standard FITS format following the definition of NOST standard 100-1.0,[43] and so offer the advantages of a high degree of self-description and the ability to use a great deal of public domain custom software such as fv , [34]. All files conform to a relatively simple FITS implementation of either a single PRIMARY image (usually empty) and associated header containing card-image header records of the form KEYWORD=VALUE, labelled HDU=0 according to the fv convention, or a single BINARY-TABLE extension of rows and columns and associated header, labelled HDU=1. Below, the header in HDU=n is called CPRO[n].HEADER; individual header records CPRO[n].KEYWORD; the BINARY-TABLE CPRO[1].TABLE with individual columns CPRO[1].COLUMN(*). While a set of keywords used across all products describes the relevant software and calibration data releases, the complete set of input files used and other common data, most HEADER keywords and TABLE columns are product specific. The layouts of all FITS products are collected in the ISO Data Product Document (Saxton 2000, [52]) which is a useful document to use with fv for examining and understanding the products available. Record alignment often required the use of one or more fillers in TABLE records that are of no practical interest with names such as CCSH[1].CSGPFILL(*), CIER[1].FILLER6(*) or CPSL[1].SPARE(*).

Product filenames are made up of a descriptive alphanumeric root and, if necessary as is usually the case, a numerical suffix reflecting the file's applicability of 8 digits for an individual observation or 3 digits for a revolution. For example, CMOS44900303 contains one observation's mosaic and APPH449 contains summary pointing data for one revolution. The descriptive root alone, without any applicable suffix, such as APPH or CMOS, is often used in the discussion below.

The overall processing scheme is represented in Figure 5.1. Satellite data received at the ground station were assembled into Telemetry Distribution Files (TDFs), making available complete information about the satellite and its subsystems for one revolution. TDFs are not in FITS format and, thus, are not suitable for use by the general observer. Instead, they were the starting point of the processing system that ended with the ingestion in IDA of a set of quality-checked data products. Products fall into different general categories which are shown in Table 5.1 along with the applications used in the latest OLP release.

As discussed in more detail below, both TDF_First_Scan and Derive_ERD are common applications for all ISO instruments. They combine general purpose and instrument specific procedures. Both operate at TDF level. Selected data are extracted by TDF_First_Scan into low level files that are mainly for internal use by OLP later in the pipeline while scientific data are extracted, decoded and recast into ERD by Derive_ERD. ERD are the first FITS-format products available for general use and contain a complete set of raw ISOCAM data for an individual observation without calibration or conversion. Two other important tasks are performed at these early stages of the pipeline: TDF_First_Scan calculates the synchronisation between the different ISOCAM time keys that hold good for a whole revolution; and Derive_ERD calculates all the relevant pointing files. General purpose products are described elsewhere, in the ISO Handbook Volume I, [40]. Although ERD can serve, for example, as a starting point for interactive analysis with CIA, the following stage of Derive_SPD performs some useful initial data selection and reduction. It extracts status and housekeeping information; logs statistics on detector data; does the first stages of image construction; and writes the SPD. Apart from supplying data for AAC, the next stage of the pipeline described in Chapter 7, SPD is also the normal starting point for interactive analysis with CIA.

5.1.1 ISOCAM use of time keys

While the various time keys used in ISO products are discussed fully in the ISO Handbook Volume I, [40] a few ISOCAM-specific points are worth bearing in mind in the context of their main purpose of enabling synchronisation of different types of data. The ISOCAM instrument has its own clock giving the Instrument Time Key, or ITK, which is reported in the General Science Prefix, or GSCP, that forms part of every row of raw data as in CIER[1].GSCPTKEY(*), for example. One of TDF_First_Scan's jobs is to calculate a clock calibration that holds for the entire revolution that is made available in the header of the CCSH file in CCSH[0].CAMITK0 and CCSH[0].CAMUTK0. These are the values of ITK and UTK, respectively, when an internal counter is reset to zero near the beginning of a revolution. They are used to calculate the ITK of each individual CIER record

CIER[1].GSCPTKEY(*) = ITK = CCSH[0].CAMITK0+CIER[1].F2BOOTTI(*)  ! CAM prime only

   TREFUTC1=            228128772 / UTC (whole seconds since 01-01-1989)
   TREFUTC2=              5112273 / UTC (remaining fraction of second)
   TREFUTK =            301458521 / ISO Uniform Time Key (UTK)
   TREFITK =             89720116 / ISO INSTRUMENT Time Key (ITK)
   TREFITKU=     0.13999950000000 / ITK unit length in seconds