Skip to content
Snippets Groups Projects

Matrix architecture design documentation

Merged Matrix architecture design documentation
matrix-architechture into 2.8.4-DEVELOPMENT
2 unresolved threads
Merged Charlotte Hausman requested to merge matrix-architechture into 2.8.4-DEVELOPMENT
2 unresolved threads

Daniel wrote up a good usage document in the README, this is to document the design decisions that the code should support. Please point out anything I missed

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
shared/workspaces/workspaces/system/ARCHITECTURE.rst 0 → 100644
92 In order to provide our users with the most accurate data products possible, we need to make every effort to
93 ensure their processing requests succeed. As data processing results depend on the outcome of CASA, we must verify the
94 CASA version used to be as correct as possible for their requests.
95
96 What makes a CASA version valid?
97 ________________________________
98 In order for a CASA version to be valid for processing the following conditions must be met:
99
100 1. It *must* be capable of running on the current cluster's operating system.
101 - All processing currently occurs within one of NRAO's processing clusters. In the future we plan to enable fully remote processing, but at this time we are limited by the conditions of our clusters' resources. The current clusters are running RHEL8; we have had issues running 6.1.x versions on this OS. These versions are considered invalid for processing.
102
103 2. It *must* be installed and linked to the SSA home area.
104 - Even if the requested version can run on the current cluster, if it's not installed in the expected area and linked to SSA home, it doesn't exist as far as processing is concerned.
105
106 3. It *must* be able to run the requested type of processing.
107 - As time passes, CASA surfaces new processing abilities. Even if a CASA version meets the other validity criteria, if it predates the start of the requested type, it's useless, as the needed recipe isn't available. For example: EVLA Standard Imaging only began support around CASA 5.4.x. Any version predating that is invalid for Standard Imaging.
  • Daniel Nemergut
    Daniel Nemergut @dnemergu started a thread on the diff
    • shared/workspaces/workspaces/system/ARCHITECTURE.rst 0 → 100644
    130 There are known CASA versions which might have been used to produce calibrations which passed QA and were ingested,
    131 which were only later determined to have processing errors, rendering the version unsuitable. The Data Analysts maintain
    132 a list of usable CASA versions and known issues, found here: `Pipeline Version History <https://science.nrao.edu/facilities/vla/data-processing/pipeline/pipeline-version-history>`_
    133
    134 In the case where the OPV is *not* valid, **the Matrix shall fallback to the CAPO defined default for the requested
    135 processing type**.
    136
    137 If the CAPO defined default is *also* not valid, **the Matrix shall fallback to the newest available version**.
    138
    139 Concisely, the version precedence chain is as follows:
    140
    141 1. The Original Processing Version shall be preferred above all else
    142
    143 2. If the OPV is not valid, use the CAPO defined default
    144
    145 3. If the default is not valid, use the most current installed version
  • Daniel Nemergut approved this merge request

    approved this merge request

  • Charlotte Hausman mentioned in commit ed55dd7a

    mentioned in commit ed55dd7a

  • Charlotte Hausman merged

    merged

    • Please register or sign in to reply