envoys.rst

Envoy Architecture

What is an Envoy?

An Envoy is defined by the Merriam-Webster dictionary as “a person delegated to represent one government in its dealings with another”.

In the context of the Workspaces System, an Envoy is a python executable program that is delegated the task of interacting with a system external to Workspaces from within a workflow. This interaction includes making sure all required inputs are present and correct as well and launching the external system in question.

The envoys that currently exist are the :doc:`CARTA envoy <../tools/carta_envoy>`, :doc:`CASA envoy <../tools/casa_envoy>` and :doc:`ingest envoy <../tools/ingest_envoy>`.

When should I build a new Envoy?

If you are needing to interact with a system external to workspaces, i.e. a system which is not built and maintained within the workspaces project, you should create an envoy. Examples of external systems Workspaces interacts with are CASA or the ingest system.

How do I build a new Envoy?

An Envoy is essentially a bridge between Workspaces and the system of interest. As such, a Workspaces Envoy has two essential functions: Setup and Launch.

Basic Envoy Architecture

An envoy is expected to be extensible if required, i.e. it should be able to launch multiple types of calls to the system of interest. As such, envoys should typically have the following structure: - A main entry point which determines which type of call to make based on supplied parser input, this file is typically named after the envoy (unless it’s the main file for CASA Envoy which is aptly named “palaver”) - A number of typed launcher classes which handle the type specific setup (typically broken up into other classes) and then make the call to execute the external system. These classes are typically contained in a file called launchers.py but can also rely on any number of other classes for setup functionality.

Example: CASA Envoy has two main modules palaver.py, the entry point, and launchers.py, which contains the CalibrationLauncher and ImageLauncher classes. However, the package also contains the setup helper modules auditor.py and foundation.py, which audit the input files for required fields and correct for HTCondor submission, and ensure all data is in the required locations for CASA execution respectively.

What should the setup phase cover?

This is the most complex part of any envoy and typically encapsulates multiple stages.

Because the external system already exists, we cannot dictate the inputs provided for execution nor the environment needed for it to run successfully. Therefore, we need to gather all required information and ensure it’s in place before making the call to run the system.