PDS_VERSION_ID = PDS3 RECORD_TYPE = STREAM SPACECRAFT_NAME = "CASSINI ORBITER" INSTRUMENT_NAME = "COMPOSITE INFRARED SPECTROMETER" MISSION_PHASE_NAME = "SATURN TOUR" DATA_SET_ID = "CO-S-CIRS-2/3/4-REFORMATTED-V1.0" OBJECT = TEXT PUBLICATION_DATE = 2006-08-31 NOTE = "Brief tutorial on how to use the data files in this data set." END_OBJECT = TEXT END RE-FORMATTED CASSINI CIRS DATA SET USER TUTORIAL This document is organized as follows: 1. Overview 2. Introduction to CIRS Data 3. Introduction to the Cassini Timeline 4. Overview of the Re-Formatted Data Set 5. Volume Names 6. Directory and File Names 7. File Types 8. Tables and Rows 9. PDS Labels 10. ASCII File Formats 11. Binary File Formats 12. Indices 1. Overview This data set contains re-formatted data and metadata from the Cassini CIRS instrument. The data set is based on the original CIRS volumes, COCIRS_0xxx. It is designed to simplify access to the data. The files have been converted from variable-length to fixed-length formats. ASCII versions of most binary files are also provided. In addition, file boundaries have been adjusted so that the data and metadata (timing and geometry information) associated with individual Cassini activities and with individual CIRS focal planes are found within a single set of files; no file contains information from more than one activity or one focal plane. 2. Introduction to CIRS Data For full details about the CIRS instrument, see CATALOG/INST.CAT. The CIRS instrument has several unusual properties, which have made the CIRS data particularly challenging to work with: (a) The instrument has variable resolution, which means that the number of samples in a spectrum varies from one observation to the next, and sometimes within a single observation. (b) The instrument has three focal planes, referred to as FP1, FP3 and FP4. (FP2 was eliminated in a descope). Because the three focal planes typically operate simultaneously, a single observation from the instrument can generate spectra covering several different ranges of wavenumber. (c) A great deal of processing is required to go from the raw interferograms to the calibrated CIRS spectra that a typical user requires. For this reason, the CIRS team has kindly archived calibrated data (called apodized spectra) in addition to the raw data. (Note that the calibration procedure is continuing to evolve; if you want the latest, most reliable calibration of a spectrum, it would be best to contact a member of the CIRS team.) (d) Although FP1 has a single circular field of view, FP3 and FP4 each consist of ten square pixels, of which five can be read out at a given time. Each pixel captures a different region of a target, and sometimes a different target. For this reason, the CIRS team has archived extensive tables of geometric metadata along with each spectrum, describing the instantaneous field of view of each pixel. To solve the problems of working with this complicated data set, the CIRS team has developed a software package called Vanilla, and has archived the data in a format that is closely tied to the software. For example, because the spectra have different lengths, the CIRS team has adopted a binary, variable-length record format, combined with an index that describes where to find a particular spectrum within the larger data file. In addition, data from the different focal planes are interspersed within the same file. Data files themselves are organized simply by time, where a single file contains every spectrum obtained within a single interval, typically lasting 8 hours. The Vanilla software makes it relatively easy for users to retrieve a desired selection of spectra from these files. However, those who do not use Vanilla might find it difficult to access the CIRS data. See See DOCUMENT/DATASIS.PDF (or DOCUMENT/DATASIS_OCR.PDF) for further details about how the original data set has been organized. See DOCUMENT/VOLSIS.PDF for more information about how the original volumes are organized. 3. Introduction to the Cassini Timeline The Cassini timeline has been allocated in terms of specific activities, sometimes referred to as "CIMS requests." Here "CIMS" stands for the "Cassini Information Management System." A CIMS request corresponds to a particular block of time in which one or more instruments observe (usually) a single target. In the Cassini timeline, one instrument is always "prime" and defines the details of the observation, such as where the instrument points and how long it dwells at each location. Often, additional instruments are "riders" and will obtain simultaneous data from the same target. CIMS requests have identifiers such as this: CIRS_016RE_E60PHASE001_VIMS It is constructed as follows: CIRS: Indicates the instrument doing the observation, regardless of whether it is prime or a rider. 016: Indicates Cassini's orbit number RE: Abbreviation for the nominal target, in this case Ring E. E60PHASE: A brief mnemonic indicating the purpose of the observation, in this case to gather the phase curve information around 60 degrees. 001: For the first activity of this type in the given orbit of Cassini. VIMS: Finally, this indicates that the VIMS (Visual and Infrared Mapping Spectrometer) instrument is prime for the observation. If CIRS had been prime, the identifier would have ended with PRIME instead. (Note that the CIMS request ID associated with a given file can be found in the PDS label as the value for the keyword OBSERVATION_ID.) 4. Overview of the Re-Formatted Data Set This data set has been designed to ease the access to the CIRS data set for scientists who do not use Vanilla. It simplifies the selection of data subsets based on target, CIMS request or CIRS focal plane. The reorganization also eliminates the need for variable-length files, on the assumption that users will have a much easier time writing programs to read fixed-length records. The re-formatted data set has the following properties. (a) File boundaries correspond to CIMS requests, so the data associated with individual Cassini activities are found within a single set of files; no file contains information from more than one CIMS activity. (b) Data files have been divided up by focal plane. (c) Metadata files (mainly timing and geometry) have been restructured to contain the identical record sequence as that in the corresponding data files. Users will find an exact 1:1 correspondence between the records in each data file and the records in the metadata files. (d) All information except the actual thermal spectra are available in ASCII format. Most users find ASCII files easier to read than binary files. However, because the spectra are quite large, they are only archived in binary. 5. Volume Names The original CIRS data volumes are named COCIRS_yymm, where yy is the last two digits of the year and mm is the month number (01 through 12). These re-formatted volumes are also organized by month. To distinguish the two data sets, re-formatted volumes have 50 added to the year. For example, the data in original volume COCIRS_0407 can be found, in the alternative form, in volume COCIRS_5407. 6. Directory and File Names The DATA directory on this volume contains five subdirectories, each of which contains a particular type of file: APODSPEC apodized, i.e. calibrated and resampled, spectra. GEODATA position and viewing geometry of the planet and moons. POIDATA pointing geometry on the planet or moon. RINDATA pointing geometry on the ring system. TARDATA listing of target bodies captured. Data files are generally named as follows: aaayymmddhhmm_FPn.* aaa a prefix indicating the file type: ISPM = interpolated spectrum (APODSPEC subdirectory); GEO = system geometry (GEODATA subdirectory); POI = planet and satellite pointing (POIDATA subdirectory); RIN = ring pointing (RINDATA subdirectory); TAR = target body listing (TARDATA subdirectory). yymmdd two-digit year, month and day when the observation began. hhmm two-digit hour and approximate minute (UTC) when the observation began. n CIRS focal plane number: 1, 3 or 4. Note that the prefixes ISPM, GEO, POI, RIN and TAR have direct analogs to the files in the original CIRS volumes. The ISPM files are found in the DATA/APODSPEC subdirectories and the others are found in the DATA/NAV_DATA subdirectories. GEO files contain an additional suffix.: GEOyymmddhhmm_FPn_bbb.* bbb the NAIF body ID indicating which moon's position and viewing geometry is described by the file. NAIF IDs are assigned as follows: 699 = Saturn 601 = Mimas 602 = Enceladus 603 = Tethys 604 = Dione 605 = Rhea 606 = Titan 607 = Hyperion 608 = Iapetus 609 = Phoebe 610 = Janus 611 = Epimetheus 612 = Helene 613 = Telesto 614 = Calypso 615 = Atlas 616 = Prometheus 617 = Pandora 618 = Pan Some binary ISPM files (in the APODSPEC subdirectory) contain an additional suffix: ISPMyymmddhhmm_FPnx.* x A, B, C, .... The suffix changes each time the spectral resolution changes in the middle of a single CIMS request. In most cases, the resolution stays fixed for an entire request, in which case no suffix appears. This suffix is necessary to avoid using variable-length data files. 7. File Types We refer to the ISPM files as data files and the GEO, POI, RIN and TAR files as metadata files. Most files come in matched pairs, consisting of one ASCII table (*.TAB) and one binary table (*.DAT). These are described by the same combined-detached PDS label *.LBL. In general, the ASCII and binary files contain exactly the same information. The ASCII file may be simpler for the user to read but the binary file contains potentially more precise numeric information. Here is an example: RIN0407020156_FP3.DAT: binary table; RIN0407020156_FP3.TAB: ASCII table; RIN0407020156_FP3.LBL: combined-detached PDS label. In addition, you will need the following format information to interpret the files: RIN_ASCII.FMT: PDS 'include' file describing the columns and format of the ASCII file. RIN_BINARY.FMT: PDS 'include' file describing the columns and format of the binary file. They are found in the same subdirectory as the data files. As discussed below, the PDS labels contain a complete, detailed description of the format of each file. The ISPM files are somewhat exceptional. The spectra are only found in the binary files; the ASCII files APODSPEC/ISPM*.TAB contain the header information about each CIRS spectrum but do not contain the spectra themselves. Also, although the binary ISPM files are occasionally broken up into pieces with suffixes A, B, C, etc., the ASCII file concatenates the information for the entire activity. (A column in the ASCII file indicates which binary file contains the corresponding spectrum). Here is an example: ISPM0407020930_FP1.TAB: ASCII table describing the whole observation. ISPM0407020930_FP1.LBL: PDS label for the above. ISPM0407020930_FP1A.DAT: binary table containing the first part of the data. ISPM0407020930_FP1A.LBL: PDS label for the above. ISPM0407020930_FP1B.DAT: binary table containing the second part of the data. ISPM0407020930_FP1B.LBL: PDS label for the above. 8. Tables and Rows Every CIRS data and metadata file is organized as a table. Each row of the table (corresponding to a record of the data file) refers to a single CIRS spectrum. CIRS spectra are identified by two parameters, the time when the spectrum was obtained and the detector (pixel) number that obtained it. The combination of time and detector number constitutes what is often referred to as the "primary key" of the table---it is unique within each file and it increases monotonically, most rapidly by detector number and then by time. The CIRS team (via the Vanilla software) has adopted the Unix time system as the time tag for each row in a table. This system counts the number of elapsed seconds since January 1, 1970, NEGLECTING leap seconds. The Unix subroutine library contains routines that convert between the Unix time as an integer and the actual UTC time. The typical user will not need to use this subroutine because the ISPM files also tabulate the time information as year, month, day, hour, minute and second. Nevertheless, the first column in each table contains the Unix time tag as an integer. The second column in each row contains the detector number. Detector numbers are defined in the file DATA/DATASIS.PDF and DATA/DATASIS_OCR.PDF, Table 1. To summarize, detector 0 refers to FP1, detectors 1-20 refer to the pixels of FP3 in their various operating modes, and detectors 21-40 refer to the pixels of FP4 in their various modes. The detector number is found in the second column of each table. These files have been organized such that the sequence of primary keys is IDENTICAL for the files associated with a particular CIMS request and focal plane. For example, the file DATA/APODSPEC/ISPM0407020156_FP3.TAB contains a particular sequence of times and detector numbers. The corresponding metadata files, DATA/POIDATA/POI0407020156_FP3.TAB and .DAT DATA/RINDATA/RIN0407020156_FP3.TAB and .DAT DATA/TARDATA/TAR0407020156_FP3.TAB and .DAT contain the exact same sequence. Thus, a user can open a set of files and read them together, record by record, with the guarantee that the primary key will always match and that the metadata will always describe the corresponding spectrum. Note that, on rare occasions, metadata was missing for a particular sequence of spectra. However, the new metadata files still contain a record, but the missing information is indicated by unphysical values of -200 in most columns. The GEO files are exceptional in that they define the geometry of the Saturn system as a function of time. The files contain, for example, the position and lighting geometry of each body during the time span of a request. Because this information depends on time but not detector number, the files contain a single record per time tag and do not contain a column of detector numbers. In practice, when using FP3 and FP4 data, one would read one record of these files for every five records of the corresponding ISPM, POI, RIN and TAR files. 9. PDS Labels PDS labels are ASCII-format text files that can be easily read via most word processing or text editing programs. Most lines are in the form keyword = value where keywords are defined in the PDS Data Dictionary; see http://pds.jpl.nasa.gov/tools/software_download.cfm . More information about labels can be found in the PDS Standards Reference; see http://pds.jpl.nasa.gov/documents/sr/ . Because all CIRS files are tables, the labels contain a "table object" indicated by the line OBJECT = ASCII_TABLE or OBJECT = BINARY_TABLE Between this line and the corresponding line END_OBJECT = ASCII_TABLE or BINARY_TABLE is a description of the table. It indicates the number of rows and columns and the range of primary key values from the first and last rows. It also contains a line ^STRUCTURE = "filename.FMT" which functions as a kind of "include" file for the label. The referenced "format file" *.FMT contains a detailed description of each column in the file. For example, OBJECT = COLUMN NAME = SCET DATA_TYPE = ASCII_INTEGER START_BYTE = 1 BYTES = 10 FORMAT = "I10" DESCRIPTION = "Spacecraft event time, encoded as an integer." UNIT = SECOND END_OBJECT = COLUMN indicates that this ASCII-format file contains the spacecraft event time or "SCET" in its first column, encoded as a Unix integer time. The column occupies the first ten bytes of the record and has units of seconds. Subsequence COLUMN objects in the format file describe the remaining columns. Note that the PDS labels contain a wealth of useful information about the corresponding file. Keywords indicate the CIMS request name as the value of the OBSERVATION_ID keyword. The RIN labels indicate the rough range of ring geometry parameters sampled by the observation; see, for example, MINIMUM_RING_RADIUS and MAXIMUM_RING_RADIUS. The POI labels indicate the rough range of surface geometry parameters, including the minimum and maximum latitude and longitude. The ISPM labels contain additional information about the pixel readout mode and the spectral resolution. 10. ASCII File Formats All data files, whether binary or ASCII, contain fixed-length records; see the keyword RECORD_BYTES near the top of each PDS label to find out the record size of the corresponding file. In the case of the ASCII files, this information is not strictly necessary because the records also contain line terminators. Note, however, that different popular operating systems use different terminators. On Unix systems, lines are terminated by a (linefeed character, control-J, ASCII 10). In some Macintosh files, lines are terminated by a (carriage return character, control-M, ASCII 13). On PCs, lines are terminated by a pair. PDS standards require all text files to use line termination. If the PDS label indicates INTERCHANGE_FORMAT = ASCII or RECORD_TYPE = STREAM, then this line termination is in use. Many word processors and text editors can figure out which type of line termination is in use, rendering this issue invisible to the user. However, some user-written computer programs may not behave quite as expected on Unix systems because of the superfluous character prior to the . If you encounter this situation, the Unix "tr" (translate) command can be used to change carriage returns to blanks: tr "\015" " " newfile.txt After executing this command, the new, translated file should read properly. 11. Binary File Formats Binary files do not contain internal line terminators, so you need to know the size of each record prior to opening a binary file. Thus, you should consult the RECORD_BYTES keyword in the PDS label. Note, however, that the binary files of metadata are very consistent: GEO*.DAT: RECORD_BYTES = 184 POI*.DAT: RECORD_BYTES = 752 RIN*.DAT: RECORD_BYTES = 512 TAR*.DAT: RECORD_BYTES = 40 However, the length of the spectral data in the ISPM*.DAT files does vary from one file to the next, due to the variable spectral resolution that the CIRS instrument can obtain. All binary files are in IEEE format with most-significant byte (MSB) first. This is the native format on Sun and PowerPC Macintosh platforms. For users on other platforms such as those with Intel-based CPUs, the native byte ordering is least-significant byte (LSB) first instead. This means that the byte ordering in each binary value will need to be swapped before the number can be used. In FORTRAN, a file can be opened with the specification CONVERT = 'LITTLE_ENDIAN' and the numbers will be converted with no further effort. For other languages, consult the .FMT file to understand the structure of each record and to determine how the byte-swapping should be carried out. Note that four-byte values must be reversed in sets of four, whereas eight-byte values must be reversed in sets of eight. If the above is too confusing, use the ASCII format files instead. Regardless, you will still need to byte-swap the spectra, which only appear in the binary ISPM files. Those are exclusively four-byte values, so byte-swapping those records exclusively in groups of four will work correctly. 12. Indices The master index file INDEX/INDEX.TAB contains the general information about every data file on the volume. Supplemental index files INDEX/RININDEX.TAB, INDEX/POIINDEX.TAB and INDEX/ISPMINDEX.TAB contain tables of these additional parameters. They can be used for identifying, for example, which observation captured a particular latitude, longitude and incidence angle on the surface of Enceladus. More detailed searches can be conducted by reading the metadata files row by row, which will enable you to identify the specific detectors that captured a target under the desired circumstances. Note that, like data files, index files have PDS labels. Those labels refer to .FMT files that describe the column structure in each index. The PDS Rings Node is developing a more sophisticated tool for selecting CIRS files based on geometric constraints. As of this writing (August 2006), however, that tool is not yet available. See http://pds-rings.seti.org/catalog/ for the latest information.