README.md

CASA Envoy: The Workspaces CASA Processing System

CASA Envoy is responsible for environment setup and launch of all types of CASA processing for the Workspaces System. Currently, this includes Standard Calibration, CMS Restores, Standard CMS Imaging, and CMS Restore-and-Image.

usage: casa_envoy [-h] [-c STANDARD_CAL STANDARD_CAL] [-i STANDARD_IMG STANDARD_IMG] [--restore] [--integrated]

Workspaces CASA processing launcher

optional arguments:
  -h, --help            show this help message and exit
  -c, --standard-cal    run the standard calibration CASA pipeline
  -i, --standard-img    run the standard imaging CASA pipeline
  --restore             run the restore measurement set CASA pipeline, use in conjunction with '-c'
  --integrated          run an integrated calibration-imaging pipeline, use in conjunction with '-i'

While initiating CASA is relatively straight forward and essentially identical between use cases, there are variations in the required PPR structure and the initial location of input data. For this reason CASA Envoy's functionality can be broken into two underlying parts: Setup and Launch.

Setup

Setup can be further divided into three essential parts: Auditing, Environment, and Data

Auditing

There are two types of auditing that occur in CASA Envoy. The first, Directory Auditing, ensures that the required directory structure of rawdata, working, and products is contained in the processing directory.

The second type is File Auditing. CASA Envoy must be provided two different files at inital call, metadata.json and PPR.xml. These files must be audited to ensure that all required fields are present. PPR.xml must be submitted directly to CASA, so it is important to make sure that it is structured correctly before submission. While metadata.json is not submitted to CASA, it does contain all the information for results delivery post-CASA and needs to be audited as well.

A further function of File Auditing is correcting PPR.xml for processing with HTCondor. CASA requires the name of the processing directory it is running in - unfortunately, with HTCondor it isn't possible to know that name prior to submitting a condor job. Therefore, PPR.xml is corrected after submission to condor with the correct directory information. This corrected PPR.xml is placed into the working directory where it is then used by CASA, and the unaltered original remains in the parent processing directory.