Using the default WS staging area instead of the capo property

Reverting the commit I previously did that was telling ingest to use the vlbiStagingDirectory capo property instead of the default stagingDirectory one.

There is now an ObservationCollector that copies everything from the drop location (e.g. difx directory) to the staging directory.

apps/cli/executables/pexable/ingest_envoy/ingest_envoy/collectors.py

@@ -144,7 +144,7 @@ class SECICollector(CollectorIF):
         :return:
         """
         self.logger.info("Collecting SECI products for staging...")
-        cache_path = self.parameters["seciCachePath"]
+        cache_path = self.parameters["cachePath"]
         workflow_dir = self.parameters["workflowDir"]
         staging_dir = self.parameters["staging_area"]
         tar_name = self.create_artifacts_name()
@@ -159,3 +159,31 @@ class SECICollector(CollectorIF):
             exit(1)
 
         aux_file_to_staging(self.staging_source_dir)
+
+
+class ObservationCollector(CollectorIF):
+    """Collect Observation Products for Ingestion"""
+
+    def __init__(self, parameters):
+        self.logger = logging.getLogger("ingest_envoy")
+        self.parameters = parameters
+        self.staging_source_dir = self.parameters["staging_area"] + "/" + self.parameters["workflowDir"]
+
+    def collect_products(self):
+        """
+        Run observation product collection script
+        """
+        self.logger.info("Collecting observation products for staging...")
+        cache_path = self.parameters["cachePath"]
+        workflow_dir = self.parameters["workflowDir"]
+        staging_dir = self.parameters["staging_area"]
+
+        # run script
+        collector = subprocess.run(
+            ["./observation-product-collector.sh", cache_path, workflow_dir, staging_dir],
+            stdout=sys.stdout,
+            stderr=sys.stderr,
+        )
+        if collector.returncode != 0:
+            self.logger.error("ERROR: Observation product collection failed!")
+            exit(1)

[Charlotte Hausman]

If you want to generalize the previously seci only cachePath parameter, I'd suggest renaming it entirely to something like sourcePath or sourceLocation instead. You're not accessing a cache area here and it's sort of confusing

[Daniel Nemergut]

I like it, renamed to sourcePath.