2.1   Normal Operations¶

Here we describe how the Header Service should work during normal operations.

1. The Header Service will run at the summit and thus will always have access to the live telemetry and other feeds from DDS. It will subscribe to all of channels of interest to build an appropriate LSST header.
2. The Header Service will run inside the EFD cluster farm at the summit and will be considered a service within the critical operations enclave. Hence it should fulfill similar requirements of performance and availability as the DAQ, EFD and OCS.
3. Most likely at a camera readout Event (exact event TBD) the Header Service formats the metadata information collected during the exposure and generates a FITS header file per CCD. This is what the EFD refers to as a large file object (LFO).
4. The Header Service client publishes the existence of the LFO via DDS. This file should be named accordingly using the imageId (or ImageName) as part of the name using an Event like LargeFileObjectAvailable.
5. The EFD automatically listens to all LFO announcement Events. When it sees one, it retrieves the file (protocol needs to be defined, but a URL is already included in the LFO announcement), and stores it in the EFD LFO annex.
6. The EFD publishes an LFO announcement that it has the header file. Both summit and base EFD’s will have a copy of the new header at this point. In the case of a communications outage T&S will sync the EFD’s after summit/base are re-connected.
7. The DMCS retrieves the header from the EFD LFO annex at the Base using its imageID and a standard filesystem path to locate it.

2.2   Catchup Mode¶

This stage represents the operational mode after a communication outage between the summit and the base.

1. Once connection is re-established between summit and base the EFD tables and LFO annex files are synced.
2. DMCS can now retrieve all previously unprocessed or non archived images from the DAQ and using their imageID retrieve the already constructed FITS header as in step 7 in the previous section.

3.1   Data Sources¶

• In the current design a large fraction of the key/value pairs for the image headers will originate from static data that will come from configuration files, such as header templates that will be specific to each instrument. These can be initialized and/or defined via DDS/SAL events.
• Telemetry and Event data from the Camera Control System (CCS), Scheduler and the Telescope Control System (TCS) delivered via DDS. The Header Service will subscribe to a number of telemetry topics from which it will update key/value pairs with real-time values from DDS streams. Examples of this type include information regarding FILTER, TELRA, TELDEC, ImageID and visitID.
• Some additional meta-data not present in the telemetry or in the static data will have to be computed on-the-fly based on telemetry or other current information. The image AIRMASS is an example of this type of meta-data. This is expected to be a light-weight computation. Some of this additional information will be critical for production while the rest can be considered courtesy meta-data.
• TBD

3.2   Data Delivery and Interface¶

The method for passing data from the Header to each Forwarder is not yet final.

• The default method will be via a message. The Header Service will publish an Event like LargeFileObjectAvailable to DDS and the existence of a Large File Object (LFO) that will contain the location and name and the file that was inserted into the EFD LFO Annex.

• The Forwarders (DMCS) will retrieve the header from the EFD LFO annex using the ImageId and a standard filesystem path to locate it.

  

  Figure 3 Header Service diagram using Archimate modeling language for the the Spectrograph.

4.1   Primary concerns¶

• In the current design, the Header Service will gather the dynamic meta-data from telemetry streams at either the beginning of the integration, at the end of the readout of an exposure, or at both times. It is unlikely that more granular (in time) telemetry is needed for Prompt Processing or archiving.
• If the above statement is incorrect, we need to investigate in which cases this will not be satisfied and gather requirements for the amount of granularity required.
• We do not have set of required key/value pairs for header meta-data. It is important that before or during the design review we establish a procedure to define the content definition of the header files for all instruments that will be supported. We propose to start with an initial list based on standard key/value pairs from recent astronomy surveys that should be augmented to satisfy at a minimum the DM software and LDF production requirements.

4.2   Secondary concerns¶

• In the current design description the EFD retrieves the file published by the Header Service. Alternatively the Header Service could write directly into the Large File Annex filesystem.
• In the current design description the EFD publishes the path with the location of the new header file(s). Alternatively the DMCS could retrieved the header file as soon as the Header Service publishes the existence of the LFO. This would require that the DMCS have read access to the Large File Annex filesystem.
• Some light-weighted meta-data required for archiving or prompt processing will not be provided by the telemetry emanating from SCS or configuration files. We plan to compute these and additional courtesy meta-data on-the-fly and insert them to the headers. Examples of such cases are information regarding AIRMASS, NITE (as a string) and a default plate solution based on the telescope pointing.
• If the delivery method described above is not performant for L1 Archiving and Prompt Processing, an alternative can be explored where the Header Client could pass fitsio FITSHDR Python Objects directly to the DMCS Forwarders.

5.1   Specifications¶

Here are the set of minimum specifications that are needed for a full Design of the Header Service.

1. Header definition for L1 Test Stand
2. Header definition for Spectrograph
3. Header definition for ComCam
4. Header definition for LSSTCam
5. Capture time definition for key/pairs values for L1 Test Stand
6. Capture time definition for key/pairs values for Spectrograph
7. Capture time definition for key/pairs values for ComCam
8. Capture time definition for key/pairs values for LSSTCam

5.2   Requirements¶

Here we present a list of proposed requirements, test and validation of those.

Figure 4 Requirements Validation Matrix