Refactor wf_monitor to reduce fd usage and improve general design

This design reduces the number of file descriptors in use down to about 1/3 of the current volume.

This code has been tested extensively locally for single job workflows and DAG workflows. It performs correctly for both the case when the log files are found and the case where we timeout looking for them.

This is in a Draft state until the tests are fixed and/or replaced

This now has tests - I did generate a lot of these with AI, so I'm willing to debate their brilliance or lack there of. There are tests for basically everything expect the really complex log reading and dag queue merging. Those can be a work in progress for now.