terry.mcguinness requested to merge workflow_actions into develop Nov 28, 2023

Created by: TerrenceMcGuinness-NOAA

Description

Updates to CI robustness for reverting back to CI-Ready from any given state

New Features:

Improved scancel routine (refactored into bash "subroutine")
Improved messaging (see below) when ever a user changes state
Any and all previous build scripts and running experiments are killed as a result of reset to Ready

Resolves #2060 Resolves #2067
Bug fix (Resetting CI tests had lingering builds simultaneously running)
New feature (Robust Resetting with messages)
Maintenance (code refactor - use of BASH subroutines in main drivers)

Example of output when user updates state and request a restart:

CI Update on Hera at 11/28/23 09:33:11 PM
=================================================
PR:201 Reset to Hera-Ready by user and is now restarting CI tests
Driver PID: 302744 on hfe12 is no longer running this test
Driver PID: has restarted as 163669 on hfe12
No current experiments to cancel in PR: 201 on Hera

Change characteristics

Is this a breaking change (a change in existing functionality)? NO
Does this change require a documentation update? NO

How has this been tested?

Reset at all steps in dev Cron CI space with appropriate kills taking place on Hera and Orion

Checklist

Any dependent changes have been merged and published
My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes generate no new warnings
New and existing tests pass with my changes
I have made corresponding changes to the documentation if necessary