Skip to content
Snippets Groups Projects
Commit 83289679 authored by Daniel Nemergut's avatar Daniel Nemergut
Browse files

Lustre locking this workflow as well since we anticipate large data sets

parent 30d781fa
No related branches found
No related tags found
2 merge requests!1605Merge 2.8.2.3 work to main,!1493WS-1893 curator workflow
Pipeline #12416 passed
......@@ -24,16 +24,11 @@ output = curator.out
error = curator.err
log = condor.log
SBIN_PATH = /lustre/aoc/cluster/pipeline/$ENV(CAPO_PROFILE)/workspaces/sbin
SPOOL_DIR = {{spool_dir}}
should_transfer_files = yes
transfer_input_files = $ENV(HOME)/.ssh/condor_transfer, nraorsync://$(SBIN_PATH)/pycapo, nraorsync://$(SBIN_PATH)/ingest_envoy, nraorsync://$(SBIN_PATH)/curator, nraorsync://$(SBIN_PATH)/productfetcher, ./metadata.json
should_transfer_files = NO
+WantIOProxy = True
request_memory = {{ramInGb}}
getenv = True
environment = "CAPO_PATH=/home/casa/capo"
requirements = HasLustre == True
queue
......@@ -43,7 +38,9 @@ queue
curator_sh = """#!/bin/sh
set -o errexit
./ingest_envoy curate --$2 $1
SBIN_PATH=/lustre/aoc/cluster/pipeline/$CAPO_PROFILE/workspaces/sbin
${SBIN_PATH}/ingest_envoy curate --$2 $1
"""
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment