No Story: Ingest manifest marshmallow
This switches ingest_envoy
to use marshmallow schemas to generate the ingestion manifest JSON files. This works by defining a schema class for each manifest component (e.g. AncillaryProductSchema
for AncillaryProduct
objects). The schema classes define the fields they will output by specifying attributes like the key (optional), type, required, nullable, etc. Field validation then happens when schema.dumps()
is called to output the JSON. Note that validation doesn't normally happen for marshmallow's dumping, only loading objects from JSON. I had to add a hook to validate our dumps.
Some of our previous validations now live in the schema classes but some of them still lives on the objects (e.g. setting destination_dir = staging_source_dir
).