README.md

Workspaces Workflow Monitor

Reads in an HTCondor log file and parses the relevant data from it, splitting it into events and formatting those into messages that are sent to the workspaces.workflow-service.workflow-status RabbitMQ exchange.

usage: wf_monitor [-h] [-P PROFILE] log_path

Workspaces workflow monitor, version 4.0.0a1.dev1. Monitor execution of a workflow from the command line.

positional arguments:
  log_path              path to the workflow's HTCondor log

optional arguments:
  -h, --help            show this help message and exit
  -P PROFILE, --profile PROFILE
                        profile name to use; e.g. test, production; DEFAULT: nmtest

Design

• Reads in HTCondor logs
  • HTCondor executes a job
  • condor-submit executed
  • Log file is created and written to a location
  • Meanwhile, wf_monitor is checking that location for the appearance of a log, sleeping between attempts
  • Once log file is found, its contents are read in
• Parses them and converts them into an AMQP event (to be determined)
  • Log file contents are parsed and relevant details extracted (relevant details TBD)
  • Those relevant details are organized in a data structure
  • An AMQPEvent is then constructed from that data structure
• AMQPEvent is given to events/pymygdala/some currently nonexistant service