diff --git a/apps/cli/executables/pexable/casa_envoy/README.md b/apps/cli/executables/pexable/casa_envoy/README.md index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..e942724f4de75cd5632f58178832761b911ce713 100644 --- a/apps/cli/executables/pexable/casa_envoy/README.md +++ b/apps/cli/executables/pexable/casa_envoy/README.md @@ -0,0 +1,66 @@ +# CASA Envoy: The Workspaces CASA Processing System + +CASA Envoy is responsible for environment setup and launch of all types of CASA processing for the Workspaces System. +Currently, this includes Standard Calibration, CMS Restores, Standard CMS Imaging, and CMS Restore-and-Image. + +``` +usage: casa_envoy [-h] [-c STANDARD_CAL STANDARD_CAL] [-i STANDARD_IMG STANDARD_IMG] [--restore] [--integrated] + +Workspaces CASA processing launcher + +optional arguments: + -h, --help show this help message and exit + -c, --standard-cal run the standard calibration CASA pipeline + -i, --standard-img run the standard imaging CASA pipeline + --restore run the restore measurement set CASA pipeline, use in conjunction with '-c' + --integrated run an integrated calibration-imaging pipeline, use in conjunction with '-i' + +``` + +While initiating CASA is relatively straight forward and essentially identical between use cases, there are variations in +the required PPR structure and the initial location of input data. For this reason CASA Envoy's functionality +can be broken into two underlying parts: Setup and Launch. + +## Setup +Setup can be further divided into three essential parts: Auditing, Environment, and Data + +### Auditing +There are two types of auditing that occur in CASA Envoy. The first, Directory Auditing, ensures that the required +directory structure of *rawdata*, *working*, and *products* is contained in the processing directory. + +The second type is File Auditing. CASA Envoy must be provided to different files at inital call, *metadata.json* and *PPR.xml*. These files must be +audited to ensure that all required fields are present. *PPR.xml* must be submitted directly to CASA so it is important +to make sure that it is sstructured correctly before submission. While *metadata.json* is not submitted to CASA, it does +contain all the information for results delivery post CASA and needs to be audited as well. + +A further function of the File Auditing is correcting *PPR.xml* for processing with HTCondor. CASA requires the name of +the processing directory it is running in - unfortunately, with HTCondor it isn't possible to know that name prior to +submitting a condor job. Therefore, *PPR.xml* is corrected after submission to condor with the correct directory +information. This corrected *PPR.xml* is placed into the *working* directory where it is then used by CASA, and the +unaltered original remains in the parent processing directory. + +### Environment +CASA has several environment variable that are required to be set by whatever system it is running on. +These include: + +| ENV Variables | Description | +|--- |--- | +| SCIPIPE_ROOT | location of processing directory | +| CASA_HOME | location of CASA version to use for processing | +| PPR_FILENAME | name of PPR file (Should always be PPR.xml, but sometimes might not) | +| LANG | Turns out condor doesn't set this right for CASA happiness so we're fixing that here | + +### Data +Usually the starting location for data is *rawdata*. This is the location where data is always placed by the +Workspaces Product Fetcher. For Standard Calibrations and Standard CMS Imaging this is correct. However, for +calibration restores, the calibration tar file needs to be extracted into the *products* directory. This step ensures +that all input data is in the correct location pre-processing for CASA to find when required. + +## Launch +Since the only difference between calibration and imaging processing is the contents of *PPR.xml*, CASA Envoy contains +a single CASA Launcher class which is utilized by the two type launcher classes: Calibration Launcher and Imaging +Launcher. Each type launcher handles both standard and restore or integration types of processing. The two type classes ensure that setup is correct for each product type, as described above, and then calls +the CASA Launcher. +<br/> +<br/> +Post CASA processing, the casa log is checked for error flags and CASA envoy exits.