>
>Evaluation of PDS PPS Data Archive
>
>by Amara Graps
>amara.graps@mpi-hd.mpg.de or amara@amara.com
>Heidelberg Cosmic Dust Group
>Max-Planck-Institut fŸr Kernphysik
>Heidelberg, Germany
>August 2002
>
>* Data Contents
>
>AAREADME.TXT
>CALIB/
>CATALOG/
>DOCUMENT/
>EASYDATA/
>EDITDATA/
>ERRATA.TXT
>FOVMAPS/
>GEOMETRY/
>IMAGES/
>INDEX/
>JITTER/
>NOISDATA/
>RAWDATA/
>SOFTWARE/
>SORCDATA/
>SPICE/
>VOLDESC.CAT
>
>
>In this file:
>
>Evaluation of the Data Contents.
>
>===============================================================================
>
>Evaluation of Data Contents of VG_2801
>
>
>">" = text written in the VG_2801 CD
>
>If I didn't say any words about it, then it means that I didn't
>read and/or evaluate it.
>
>***********************************
>AAREADME.TXT
>***********************************
>
>This file is the most important file on the disk. The text in this
>file first introduces the user to the the PPS archive, it provides
>the volume format, it gives information for the data file formats,
>and shows the organization of the data on the CD in a useful tree
>structure.
>
>***********************************
>Directory CALIB/
>***********************************
>
>CALINFO.TXT Comments
>---------------------
>
>>A set of conventions are used for all PPS calibration file names.
>
>I like very much this labelling format.
>
>>Since multiple solutions have been used in the past (and may be
>>used in the future), this convention permits more than one
>>calibration to be provided
>
>I think is a very nice way for flexibility to be built into present
>and future sets of calibration data.
>
>==============================================================
>PS1 (Saturn) Calibration files
>==============================================================
>
>A comment. I have a table, Notebook #4, pg. 127,  in Lonne's
>handwriting, that shows star calibrations from 1981: He lists an
>alpha Lyr, an alpha Cma, and an alpha Leo star calibration on Day
>306 (1981). I can tell you the FDS times. It looks like ALPHA_LYR,
>ALPHA_CMA and ALPHA_LEO were not recovered for this 1981, Day 306?
>(this question is more to Karen Simmons and Lonne Lane, since I
>never had these in my hand, or used it.)

The data archive contains all the star calibration files I was able to
get from Amara Graps and from Karen Simmons. If additional files turn
up at some point, we can archive them in a supplemental data set (or a
new version of this one). However, it seems unlikely to me that any
unique, critical information has been lost due to missing star
calibrations.

>
>More comments about the Saturn Calibration files in the
>Directory RAW section.
>
>PS1CO1.TAB   (LWE calibration curve data)
>-------------------------------------
>
>Looking at the numbers, checking from notes in my "Notebook #3
>September 1983.
>
> From 23 Sept 1983 Memo by LWE regarding the PPS calibration:
>
>*64,689-79,300 km, PROCCS 132-354
>bkg = 8.73+15.567*exp(-8.528E-3*(PROCCS-132))
>
>*79,300-120,200 km, PROCCS 355-994
>bkg=11.71-1.807E-3*PROCCS
>
>*120,200-142,200 km, PROCCS 995-1388
>bkg=24.48-1.466E-2*PROCCS
>
>Checking some numbers:
>REC   Data     Star      Background
>  150, 52.80193, 39.00000, 13.80193,  0.42426,  0.30000,  0.30000,  0.00000
>                           ^^^^^^^^
>bkg =  22.0818  <----
>Is the PROCCS record number the same? This bkg looks like PROCCS record ~30,
>instead of record 150.
>
>  400, 49.78211, 39.00000, 10.78211,  0.42426,  0.30000,  0.30000,  0.00000
>                           ^^^^^^^^
>bkg =  10.9872  <----
>Is the PROCCS record number the same? This bkg looks like PROCCS record ~285,
>instead of record 400.
>
>1000, 47.15609, 39.00000,  8.15609,  0.42426,  0.30000,  0.30000,  0.00000
>                            ^^^^^^^^
>bkg = 9.82000  <----
>Is the PROCCS record number the same? This bkg looks like PROCCS record ~885,
>instead of record 1000.
>
>The PROCCS record number differ by ~115 records between what I think
>they are, based on my notebook notes, and what the background
>numbers are in this file.
>
>I recall that there was introduced in later 1980s a difference
>between the "JPL PROCCS" record numbers and the "LASP PROCCS" record
>numbers because of bad data thrown away (?). My LWE memo above
>referred to the JPL PROCCS numbers, and my notebooks all use that as
>my reference point in the PPS Saturn data. I assume that you
>received these data from Karen Simmons (LASP)? Karen most likely
>incorporated the 'standard' LWE background number to a file and then
>the records changed when bad data got thrown away, but the
>background values stayed the same, so it's probably OK. Checking
>this with a region we know well.

The discrepancy arises from the fact that we have thrown away 114
records from the beginning of the file EDITDATA/PS1D01.DAT. This is
documented in the label of that file.

The label for the calibration file CALIB/PS1C01.TAB refers to the
program used to create it, DOCUMENT/CALIB/PS1C01.FOR. This program
shows how the background calibration was derived. It also shows how to
convert between new record numbers and old PROCCS record numbers.

I apologize for the confusion, which will only arise for users of the
old data trying to use the new PDS archive.

I have added a brief "Note to prior users of the PPS data" to
TUTORIAL.TXT, and also a warning in EDITDATA/DATAINFO.TXT that record
numbers have changed. I hope that this, combined with the pointers in
the file labels, will be sufficient.

As you noted below, when the same region of the rings is identified in
both data sets, the background values are found to agree. So this does
not indicate a problem with the calibration files, just confusion over
record numbers.

>
>Checking background using the Encke Gap
>-----------------------------------------
>
>Encke Gap. I have: (Notebook #4, pg. 55)
>
>PROCCS 1219, pt ~300 to PROCCS 1225, ~pt. 120
>133426.4 km to 133748.6 km  (LWE ranges)
>PROCCS 1222 has total signal value of 45.5.
>
>(Eccentric ring is PROCCS 1219, pt. 260-295.)
>
>The file that should contain the background information is
>PS1C01.TAB. Since I must first start with a known range, I can use
>the PS1GO1.TAB file to extract the (first LWE) range information,
>and from the range, get the new PROCCS record. From the PROCCS
>record, I can use PS1C01.TAB, in order to extract the background.
>
>File PS1G01 gives:
>1105,133402.75654,"44003:24:701"
>therefore my PROCCS 1219 = new PROCCS 1105
>
>File PS1C01.TAB gives:
>record   total    star     background
>1100, 45.69009, 39.00000,  6.69009,  0.42426,  0.30000,  0.30000,  0.00000
>1110, 45.54349, 39.00000,  6.54349,  0.42426,  0.30000,  0.30000,  0.00000
>
>So background = ~6.6.
>
>Now checking with the old LWE formula
>bkg=24.48-1.466E-2*PROCCS
>bkg=24.48-1.466E-2*(1219)
>bkg = 6.60946
>OK.
>
>
>
>PS2C01    (GS-3 mode background data)
>----------------------------------
>
>PS1CO2.TAB
>
>561 lines, number of points is right.
>
>Checking Values.
>
>Let:
>PROCCS = 1636 the beginning record of the GS3 data.
>scale = 53.2 = 40*1.33 = the scale for the counts timing
>
>Empirical equation from my Notebook#4, Page #130
>totalsignal =
>-8.88213E-4*(index()+PROCCS)*scale +5.7299*scale + 39.0*scale
>
>The values for column 2: "Total Signal" match.
>
>------------------------------
>
>>PS2C01.FOR
>
>>This file was generated by
>>running program DOCUMENT/CALIB/PS2C01.FOR based on source file
>>SORCDATA/OCGS40.DAT provided by Amara Graps.
>
>Nice. Thank you for archiving this file (OCGS40.DAT). The rest of
>the cardoard box with my Voyager 9-track tapes was chewed by
>termites. You saved it, in time. My notebooks survived, but with a
>couple of termite holes.
>
>------------------------------
>
>>PS2R01.FOR
>
>>This program generates file RAWDATA/PS2R01.DAT in our standard raw
>>data file format, from Amara Graps' file SORCDATA/OCGS40.DAT.  It
>>contains the only known copy of GS-3 mode data that followed
>>immediately after the PPS ring occultation experiment at Saturn.
>
>I had no idea that no one else had a copy of that data!
>
>When Karen Simmons dug it up for us, Lonne and I considered it part
>of the rest of the PPS Saturn stellar occultation. I'm very glad
>that you make this point, because some people might miss that this
>data was extremely close in time to the 'regular' PPS occultation,
>so close, in fact, that we treated it as part of the whole
>occultation.
>
>
>>The source file contains no time tags.  However, the first 1635
>>samples in the source file are OC-1 points from file
>>RAWDATA/PS1R01.DAT, resampled to match the integration time of the
>>GS-3 mode.  By matching up the two data sets, we have been able to
>>reconstruct the GS-3 time tags unambiguously.
>
>Nice work, Mark!
>
>------------------------------
>
>>PS2R01B.FOR
>
>>This task is necessary because it enables us to reconstruct the timing of
>>some true GS-3 mode data recorded nowhere else but in Amara Graps'
>>file. [...]
>>By the fact that the values in the two files
>>match, we have been able to reconstruct the timing of the GS-3 mode
>>data.
>
>This makes me proud and glad (especially since I tow(ed) those
>tapes, notebooks and boxes around with me for more than 15 years!),
>Thank you.
>
>==============================================================
>PU1 (Uranus sigma Sgr), PU2 (Uranus beta Per) Calibration files
>==============================================================
>
>Files:  PU1C01*   PU2C01*   PU1C02*   PU2C02*
>---------------------------------------------
>
>I can read the data.
>
>Files:
>
>>PU1C03E*  These particular files are based on the analysis of
>>PU2C03E*  the Epsilon Ring by Graps et al. (1994).
>                                             ^^^^^^
>                                              1995

The reference is fixed.

>PU1C03EI.TAB
>OK (matches SSI, Table 2 in Graps et al. (1995) paper)
>
>PU1C03EE.TAB
>OK (matches SSE, Table 2 in Graps et al. (1995) paper)
>
>PU2C03EI.TAB
>OK (matches BPI, Table 2 in Graps et al. (1995) paper)
>
>PU2C03EE.TAB
>OK (matches BPE, Table 2 in Graps et al. (1995) paper)
>
>------------------------------
>
>>APPENDIX
>
>>The following memo was written by Philip Nicholson and summarizes
>>many of the numeric values that went into the PPS calibration files.
>
>>======================================================================
>>Documentation on photometric calibrations used for PPS data at Uranus
>>and Neptune.
>
>>compiled by: P.D. Nicholson, 9 Dec 1993.
>>--------------------------------------------------------------------
>
>Would it be worthwhile to put in the text from Table 2 and 3 of our
>Graps, et al (Showalter, Lissauer and Kary)  1995 AJ paper? We have
>summarized those calibrations, plus added our own.
>
>I attach the TeX tables, of Table 2, 3, Graps et al (1995)
>to an Appendix at the end of this evaluation.
>

Rather than attempt to re-format the tables for inclusion, I included
a reference to this paper. The appendices are all now archived as
DOCUMENT/GRAP2002.TXT.

>
>==============================================================
>PN (Neptune) Data
>==============================================================
>
>PN1C01*,  PN1C02  (Neptune)
>
>I can read the data
>
>
>***********************************
>Directory CATALOG/
>***********************************
>
>>CATINFO.TXT  - This file.
>
>Clear.
>
>>DATASET.CAT
>
>Clear, easy to understand.
>
>>DSCOLL.CAT
>
>Clear.
>
>>INST.CAT
>
>Clear. (Lillie's instrument paper)
>
>>INSTHOST.CAT
>
>Excellent summary!
>
>>MISSION.CAT
>
>Also Excellent summary!
>
>>PERSON.CAT
>
>Looks complete.
>
>>REF.CAT
>
>Looks complete.
>
>
>***********************************
>Directory DOCUMENT/
>***********************************
>
>>DOCINFO.TXT  - This file.
>
>Clear.
>
>>PDSDD.TXT
>
>Wow! (a lot of information)
>
>>POLES.TXT
>
>Good and important.
>
>>TUTORIAL.TXT
>
>Excellent. Should give all of the information that a user
>should need.
>
>***********************************
>Directory EASYDATA/
>***********************************
>
>This is a good idea. Would you say that few people would
>care about PROCCS records? (most of my notes reference
>PROCCS instead of distance from Saturn.
>
>>DATAINFO.TXT - This file.
>
>Clear.
>
>>/KM000_1  - Directory containing ring profiles at 0.1 km
>			resolution.
>No Saturn PPS Data?

The intrinsic resolution of the data is too coarse to resample
it at 0.1 km resolution.

>
>>/KM000_2  - Directory containing ring profiles at 0.2 km
>			resolution.
>No Saturn PPS Data?

I have added the 0.2-km files, although they are a bit noisy.

>
>>/KM000_5  - Directory containing ring profiles at 0.5 km
>			resolution.
>No Saturn PPS Data?

I have added the 0.5-km files.

>
>>/KM001    - Directory containing ring profiles at 1 km
>>			resolution.
>
>I extracted Encke Gap data from  PS1P0113.TAB. Easy as
>described.
>
>
>***********************************
>Directory EDITDATA
>***********************************
>
>>These are the recommended data files to use for essentially all PPS
>>data analysis.  The Rings Node has gone to considerable effort to
>>test and identify the reliable and useful segments of the data found
>>in the RAWDATA directory.  Further information about how the data
>>files were edited may be found below and in the PDS label files; see,
>>in particular, the DESCRIPTION fields.
>
>OK.
>
>>DATAINFO.TXT - This file.
>
>Clear.
>
>>PS1D01.DAT   - Saturn delta Sco ring occultation data file.
>>PS1D01.LBL   - Detached PDS label for the above.
>
>I was wondering where were the corresponding ranges, but I see they
>are in GEOMETRY.
>
>
>5. Processing Software
>----------------------
>
>>FORTRAN-77 source code for all the programs used in the processing of
>>raw data files can be found in the DOCUMENT/EDITDATA directory.  They
>>are included here for documentation purposes only and are not
>>otherwise supported.
>
>Thank you for providing this software.
>
>The program VIEWREC1.FOR looks like a useful program to extract
>specific records from binary files in this directory and write into
>an ASCII file. Since I work on a Unix box, (Sun) there were a couple
>of editing changes I needed (to remove the VAX extensions:
>recordtype, carriagecontrol. Did I miss another extension?). However
>I didn't succeed with extracting a record of PPS data.
>
>Compiling it is OK, after those changes
>>  f77 -o viewrec1.out viewrec1.f
>viewrec1.f:
>  MAIN viewrec1:
>
>Running it, is not OK
>>  viewrec1.out
>  Enter input filename
>PS1D01.DAT
>  Type 1 for integer*4 data(600)
>  Type 2 for integer*2 data(80)
>  Type 3 for integer*1 data(600)
>3
>  Enter record number
>1000
>eor/uio: [1010] off end of record
>logical unit 1, named 'PS1D01.DAT'
>lately: reading direct unformatted external IO
>Abort
>
>Perhaps I'm missing something obvious, but do we have a program on
>this disk to extract the raw binary data in the directory EDITDATA,
>and write the output into an ASCII file, while the user is using a
>Unix box?

A new set of programs has been added to the SOFTWARE directory:
DUMPOC1.FOR, DUMPOC1.C, DUMPGS3.FOR, DUMPGS3.C.  They are C and
FORTRAN versions of simple programs to extract records from the
EDITDATA binary files.  My hope is that they will solve the problem
you and others have had reading the binary data files.

>
>***********************************
>File ERRATA.TXT
>***********************************
>
>OK
>
>***********************************
>Directory FOVMAPS/
>***********************************
>
>OK.
>
>I made a comment here originally that I have a note and image
>regarding a FOV Map, in my Notebook #4, pg. 155. However I see that
>you have already included it in Directory IMAGES.
>
>
>***********************************
>Directory GEOMETRY/
>***********************************
>
>
>>GEOMINFO.TXT - This file.
>
>Clear
>
>>The geometry files describe the ring intercept geometry of each ring
>>occultation in the data set.  Each .TAB file tabulates geometric
>>parameters of the ring intercept point as a function of record index;
>
>Since we must interpolate between the 600 points in a record, is this
>accurate enough? Is yes, is linear interpolation accurate enough?
>

With this sampling, the worst error that arises from linear
interpolation is ~1 meter. Our tools use cubic spline interpolation
which is even better.

A note near the top of GEOMETRY/GEOMINFO.TXT now makes this point.

>
>***********************************
>Directory IMAGES/
>***********************************
>
>>DATAINFO.TXT - This file.
>
>Clear.
>
>***********************************
>Directory INDEX/
>***********************************
>
>>INDXINFO.TXT
>
>Clear
>
>>INDEX.TAB    - Index of all data files in tabular format.
>>INDEX.LBL    - PDS label for INDEX.TAB.
>
>Thank you for this.
>
>***********************************
>Directory JITTER/
>***********************************
>
>>DATAINFO.TXT - This file.
>
>Clear
>
>>   PS1J01.TAB   - Saturn delta Sco jitter data file.
>>   PS1R01.LBL   - Detached PDS label for the above.
>
>I put a note about this in APPENDIX B of this evaluation.
>
>
>***********************************
>Directory NOISDATA/
>***********************************
>
>>DATAINFO.TXT - This file.
>
>Clear.
>
>>In
>>general, this editing has involved removal of all known rings from the
>>data files.  The value of these files is that they contain uncorrupted
>>measurements of background plus stellar photon counts, so they are
>>especially useful for calibration purposes.
>
>Useful!
>
>
>***********************************
>Directory RAWDATA/
>***********************************
>
>DATAINFO.TXT
>-------------
>
>>2.3 D309LMCY.LIS
>
>>This particular file contains the timing information for star
>>calibration data obtained by viewing delta Sco in GS-3 mode about two
>>months after the Voyager 2 encounter with Saturn.  The data file was
>>used by [GRAPS&LANE1986] and is identified as the "Day 309a" data set
>>in their Table I.
>
>>2.4 D309STCA.GS3
>
>>This particular file contains raw star calibration data obtained by
>>viewing delta Sco in GS-3 mode about two months after the Voyager 2
>>encounter with Saturn.  The data file was used by [GRAPS&LANE1986] and
>>is identified as the "Day 309a" data set in their Table I.
>
>You could add,that, here is the only GS-3 starcal data in which we
>have >~ 10,000 pts (25min). But only the last 2800 points are good
>(Lonne says because of instrument warmup (Notebook #5, pgs. 183).

Noted in SORCDATA/DATAINFO.TXT.

>
>You could also add that one could test correlations between limit
>cycle data with the star calibration data using these files, in
>order to note how the star was wandering in the PPS FOV (I did this
>for [GRAPS&LANE1986], and found no correlations).
>

Also noted in SORCDATA/DATAINFO.TXT.

>
>>2.9 P21120.SCO
>
>>This particular file contains the data from a delta Sco star
>>calibration experiment carried out in GS-3 mode in May 1985, almost
>>four years after the Saturn ring occultation experiment.  This file
>>was provided by Amara Graps and serves as the source file for
>>RAWDATA/PS8R01.DAT.
>
>You could add that this file looks like it has a limit cycle problem
>embedded in the data. It is probably OK to use for statistical tests
>if the sample sizes are small. The last 2000 points are the best.

Noted in SORCDATA/DATAINFO.TXT.

>
>>2.17 PESCO.DAT
>
>>This particular file contains raw star calibration data obtained by
>>viewing delta Sco in OC-1 mode about two months after the Voyager 2
>>encounter with Saturn.  The data file was used by [GRAPS&LANE1986]
>>and is identified as the "Day 309b" data set in their Table I.  It
>>was provided by Amara Graps and serves as the source file for
>>RAWDATA/PS6R01.DAT.
>
>Would it be worthwhile giving more details: (20 minutes, 199 PROCCS
>records, last record has bad values) ?

Noted in SORCDATA/DATAINFO.TXT.

>
>I would add that this was also the star calibration file used by
>[GRAPSETAL1984, Table 1, item (4)], and was generally the most (or
>_my_ most) frequently-used star calibration file for the Saturn PPS
>ring occultation dataset.

Noted in SORCDATA/DATAINFO.TXT.

>
>>2.19 S3092350.DAT
>
>>This particular file contains raw star calibration data obtained by
>>viewing delta Sco in GS-3 mode about two months after the Voyager 2
>>encounter with Saturn.  The data file was used by [GRAPS&LANE1986] and
>>is identified as the "Day 309c" data set in their Table I.  It was
>>provided by Amara Graps and serves as the source file for
>>RAWDATA/PS7R01.DAT.
>
>You could add that, since this  (9.6 min) dataset was acquired
>immediately after PESCO.DAT (above), when you joined them (binning
>properly), it provided a means to test how the PPS behaved when
>going between OC-1 mode and GS-3 mode. I had this concept in mind
>when testing the PPS stellar occultation data between OC-1 and the
>extra GS-3 mode.
>
>In this data, the first half is higher by 13 counts than the second
>half. (Notebook #4, pg. 149.)I don't know the reason why.

Noted in SORCDATA/DATAINFO.TXT.

>
>>2.20 SCO153.DAT
>
>>This particular file contains raw star calibration data obtained by
>>viewing delta Sco in GS-3 mode about 2.5 months before the Voyager 2
>>encounter with Saturn.  It was provided by Amara Graps and serves as
>>the source file for RAWDATA/PS3R01.DAT.
>
>You could add that the temperature was in transitio nwith a warm-up so
>one would need to chop of off the first 1000 pts.

Noted in SORCDATA/DATAINFO.TXT.  Thank you for all this useful
information!

>
>***********************************
>Directory SOFTWARE/
>***********************************
>
>>SOFTINFO.TXT     - This file.
>
>Clear.
>
>>OAL/ - Directory tree containing the source code and
>>         documentation for Object Access Library 1.3 and
>>         the Light Label Library 1.1.
>
>OAL is completely new to me. Is this the common or recommended way
>to access the PDS PPS EDITDATA binary files?
>
>
>***********************************
>Directory SORCDATA/
>***********************************
>
>Noted these correspondences with my old Saturn calibration files.
>I've added more comments in RAW for each of these files.
>
>OCGS40.DAT / PS2R01
>SCO153.DAT / PS3R01
>P20801.DAT /  PS4R01
>D309STCAL.GS3/D309LMCY  / PS5R01.DAT
>PESCO.DAT / PS6R01
>S3092350.DAT  / PS7R01
>P21120.SCO / PS8R01
>
>Noted these correspondences with my old Uranus calibration files.
>
>With the exception of P21936.DAT, I didn't use these files, and have
>no information about them in my notebooks (only that the files
>exist.)
>
>P21936.DAT / PU3R01
>P24604.DAT / PU4R01
>P26287.DAT / PU5R01A
>P26313.DAT / PU5R01B
>P26320.DAT / PU5R01C.DAT
>P26328.DAT / PU5R01D.DAT
>P26706.DAT / PU6R01
>
>***********************************
>Directory SPICE/
>***********************************
>
>>DATAINFO.TXT
>
>Clear.
>
>------------------APPENDICES--------------------------------------------
>

Thank you for the extensive appendices.  They have been extracted and
archived in the data set as DOCUMENT/GRAP2002.TXT.

I have cited this document in CALIB/CALINFO.TXT and JITTER/DATAINFO.TXT.