Versions Compared

Version Old Version 12 New Version 13
Changes made by

Andrew Howard

Bradford Holden (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • sub-FITS are FITS files containing images or data (or both) and are created by subsystems throughout KPF

  • Some are created by devices (i.e. Archons) and some by dispatchers (i.e. Ca HK Detector dispatcher) The dispatchers will attempt to make sure that the appropriate header data are in each sub-FITS unit.

  • Each sub-FITS is designed to be stand-alone, and contain all of the relevant subsystem information within

  • Each sub-FITS that is a multi-extension FITS file will have a primary header which has no data.

  • The sub-FITS are assembled together by the FITS Assembler to form a single ‘L0’ FITS file per observation

...

  • The sub-FITS sub-directories will hold the separate FITS from each subsystem before L0 assembly

  • The sub-FITS will not be immediately deleted after L0 assembly, but will remain in the directory for one month (TBC) as a backup

  • The actual file location for each output file will be broadcast by the dispatcher appropriate for controlling the file construction.

...

Final L0 Filename

As per KPF File Names, the L0 Filename expected by the DRP is:

...

  1. get the current obsID value from the obsID keyword

  2. create an empty L0 FITS file (with correct filename) to append to

  3. pull telemetry from the desired KTL keywords and append them to the primary header in HDU 1

  4. append the obsID to the primary header in HDU 1

  5. look in the /Green sub-FITS directory FRAMENO keyword or as a user input

  6. look in the kpfgreen.FITSFILE for a file named “KPF_Green_obsID.FITS”

    1. if it exists, append it to the L0 FITS file as per diagram above

    2. if it does not exist, append empty GREEN_AMP1, GREEN_AMP2, GREEN_AMP3, GREEN_AMP4 HDUs to the L0 FITS

    3. delete the primary HDU and place amplfier specific keywords into the each amplifier’s header, move non-amplifier specific data to the primary header for the final L0 file

  7. look in the /Red sub-FITS directory kpfred.FITSFILE for a file named “KPF_Red_obsID.FITS”

    1. if it exists, append it to the L0 FITS file as per diagram above

    2. if it does not exist, append empty RED_AMP1, RED_AMP2, RED_AMP3, RED_AMP4 HDUs to the L0 FITS. Note the number of amplifiers is variable.

    3. delete the primary HDU and place amplfier specific keywords into the each amplifier’s header, move non-amplifier specific data to the primary header for the final L0 file

  8. look in the /Ca_HK sub-FITS directory kpf_hk.FILENAME for a file named “KPF_Ca_HK_obsID.FITS”

    1. if it exists, append it to the L0 FITS file as per diagram above

    2. if it does not exist, append empty CA_HK HDU to the L0 FITS

  9. look in the /ExpMeter sub-FITS directory for a file named “KPF_ExpMeter_obsID.FITS”

    1. if it exists, append it to the L0 FITS file as per diagram above

    2. if it does not exist, append empty EXPMETER HDU to the L0 FITS

  10. look in the /GuideCam sub-FITS directory for a file named “KPF_GuideCam_obsID.FITS”

    1. if it exists, append it to the L0 FITS file as per diagram above

    2. if it does not exist, append empty GUIDECAM HDU to the L0 FITS

  11. pull telemetry from the desired KTL keywords and append them to the primary header in HDU 1 (see below)

  12. create the new HDU with primary first, then append other HDUs

  13. write the output to a user selectable directory that defaults to the above.

Notes on file locations of Ca HK and Guide Camera FITS

The Ca HK and Guide Camera sub-FITS will be created on the KPF FIU computer (an ASL Lancelot located within the K1 server room). I believe these sub-FITS will need to be transferred from there to kpf-asl-prime in order for the FITS Assembler to access them.

This copying could happen:

...

immediately after each sub-FITS is created. This is likely the preferred approach as the Ca HK detector and Guide Camera will read out faster than the main spectrometer Green and Red detectors. If the Ca HK detector dispatcher and the Guide Camera dispatcher copied their sub-FITS to kpf-asl-prime immediately after creation, they would be saved on kpf-asl-prime and ready to assemble likely before the main spectrometer CCDs have finished reading out.

...

by the FITS Assembler. Not ideal as then the FITS Assembler has to stop and wait for the files to arrive before proceeding.

...

This computer must NFS mount the /data/ partition from kpf-asl-prime to use that file system for writing out the data. That said, as long the kpf-asl-prime has access to the KPF FIU computer’s filesystem and the dispatchers broadcast through a keyword the location of the file output, the L0 assembler could find the files.

...

FITS Header Keyword Construction

The assembler will need to pull data from two different sources to construct the final primary header. The first are the individual FITS files themselves. The second source of data are values from various other subsystems that are broadcast as keywords during the exposure.

The DRP requirements on the FITS header construction are that:

  1. The contents of the primary headers of multi-extension files (such as those from the red and green cameras) be copied into the primary but in a manner that preserves their location, unless those contents are specific to a single amplifier. Those should be moved to the headers of the sub-images themselves.

  2. Each keyword in the primary header be 8 characters.

Copying FITS Header Keywords into the L0 primary header

To create new keywords in the primary header that preserve the origin of the original data when it is copied from a sub-FITS primary header, each keyword gets a two letter prefix that shows the origin of the keyword.

  1. GR - the green detector

  2. RD - the red detector

  3. HK - the Ca H&K detector

  4. EM - the exposure meter

  5. GC - the guide camera

  6. SO - the solar radiance thing

The comment for each keyword will have the original keyword and the subsystem it came from appended. For example, the time zone in the original red detector image becomes the following keyword, value, and comment combination in the final L0 image.

RDTM_ZO = 'GMT ' / time zone Keyword TM_ZONE Red

Telemetry before, during, and after the exposure

Keck and Lick Observatories rely on a postgres database that collects and records all keyword values and their time of broadcast. The keyheader program takes a timestamp and finds the last broadcast value for a given keyword at that timestamp. The program then turns that into a FITS header compatible output. A request for the kpfexpose.EXPOSE value becomes

EXPTIME = 10.0 / Requested exposure time

Because a time stamp for an event takes time to be broadcast, it is best not to search for data at the timestamp but rather a little after.

The final L0 assembler will take the kpfexpose.STARTTIME and kpfexpose.ENDTIME, offset each by a small time, and then search for the telemetry in the keyheader database around those times. Those are denoted as the time_beg and time_end in the FITSAssmeble library. Some states will only be true between those times (shutters will only be open during an exposure for example) so a third time is searched which is just the average the start and end times.

Construction of the final FITS primary header

  1. Create a blank primary header data unit which will become the final one

  2. Read in each primary header from the sub-FITS, and move keywords to the L0 primary or to other FITS extensions as appropriate.

  3. For each input file, compare the DATE-BEG and DATE-END values with those from kpfexpose.STARTTIME and kpfexpose.ENDTIME. If the kpfexpose values are significantly older than the values in the files, use the values in the files. These file values become time_beg and time_end used for queries

  4. Use keyheader to make three queries. One at time_beg plus an offset, one at the average of time_beg and time_end which is called time_mid, and finally one at time_end plus an offset.

  5. The comments are part of the keyheader configuration file and can include the dispatchers and dispatcher keywords that the data come from.