Conveyor documentation

apps/cli/executables/pexable/conveyor/README.md

-# Workspaces Standard Capability Delivery/Post QA System
+# Conveyor: The Workspaces Quality Assurance Delivery/Retrieval System
 
-Delivery system for standard calibration and standard imaging capabilities.
-It is anticipated that post QA operations for these capabilities will also occur here.
+Conveyor is responsible for delivery to, and retrieval from, the Workspaces Quality Assurance area on lustre.
+It is also responsible for maintaining the linkages between the QA directories and the Workspaces parent
+processing directory during the QA process. 
+
+Capabilities which require QA are handled differently upon workflow completion.
+Where other requests are delivered directly to the user for download, requests requiring QA
+are delivered to the QA area on lustre where they wait for a Data Analysts to inspect them. Once
+the request has been either QA Passed or QA Failed, the data is retrieved from the QA area back to the 
+Workspaces parent processing directory for either post fail operations or ingestion into the archive.
+
+Capabilities which require QA include, but are not limited to, Standard Calibration and Standard Imaging.
+
+## Usage
+
+```shell
+usage: conveyor [-h] [--deliver DELIVER] [--retrieve RETRIEVE] [--deliver-img DELIVER_IMG] [--retrieve-img RETRIEVE_IMG]
+
+Workspaces Standard Capability Postprocessing
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --deliver DELIVER     deliver standard calibration directories to analyst QA area
+  --retrieve RETRIEVE   return standard calibration directories to original parent directory
+  --deliver-img DELIVER_IMG
+                        deliver standard image results to analyst image QA area
+  --retrieve-img RETRIEVE_IMG
+                        retrieve standard image results to original parent directory
+```
+
+Conveyor only requires a single input regardless of the type of conveyance chosen - The workflow's 
+metadata.json file. This file holds all information needed to keep track of the original directories
+of the request.
+
+## QA Delivery
+
+Once a workflow is complete, if the capability has a QA step, conveyor caches and unzips the weblog tar file,
+and then moves the `rawdata`, `products`, and `working` directories from the parent processing directory in
+the Workspaces spool area into the appropriate Workspaces QA area - either `workspaces/qa2` 
+or `workspaces/image-qa`.
+
+Conveyor then creates symlinks for each moved directory between their parent processing directory 
+and the new qa directory. This allows all files produced by a workflow to remain *continuously 
+accessible no matter their location*. This was particularly important for viewing the CASA weblog
+from the request status page. 
+
+## QA Retrieval
+
+Once the DAs and/or AOD are finished with the QA process, the files and directories need to be moved back to 
+their original parent processing directory. 
+
+Conveyor first checks for and break any symlinks, and then moves the requested directories back to their original
+location in spool. This is the first stage of any ingestion workflow and will also occur for QA Fail, however that 
+is not yet implemented.
 

apps/cli/utilities/ws_metrics/README.md

+# WS Metrics: The Workspaces Metrics Utility
+
+This utility is meant to simplify, and ensure the consistency of, metrics run for the 
+Workspaces system.
+
+
+## Usage
+
+```shell
+**********************************
+* WELCOME TO WORKSPACES METRICS! *
+**********************************
+usage: ws_metrics [-h] [-c CAPABILITY] [-b BETWEEN BETWEEN BETWEEN] [-rd REQUESTDATASIZE] [-d DATASIZE DATASIZE]
+
+Workspaces metrics reporter
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -c CAPABILITY, --capability CAPABILITY
+                        report number of times a capability has executed
+  -b BETWEEN BETWEEN BETWEEN, --between BETWEEN BETWEEN BETWEEN
+                        report the number of times a capability has executed between two user provided timestamps.
+  -rd REQUESTDATASIZE, --requestdatasize REQUESTDATASIZE
+                        reports the volume of data downloaded for this request in kilobytes.
+  -d DATASIZE DATASIZE, --datasize DATASIZE DATASIZE
+                        reports the total volume of data downloaded via the workspaces system over a specified period of time.
+
+```
+
+The current metrics system is documented more throughly here:
+
+https://open-confluence.nrao.edu/display/SSA/Workspaces+Metrics+Documentation