@Marvin I am using the prefect shell to invoke some existing scripts via prefect flow. If I cancel the flow from the UI, it is not cancelling the process that was triggered by shell invoke

It seems you're facing a challenge with cancelling shell processes triggered by a Prefect flow. Prefect's cancellation mechanism may not fully terminate external processes initiated via the shell. The cancellation process in Prefect is infrastructure-dependent, meaning it can only terminate processes if they are within its control scope. If the infrastructure (like a shell process) doesn't support cancellation, Prefect might only mark the flow as cancelled without actually terminating the process. Here's a brief rundown of the situation and potential steps you might consider: 1. Infrastructure Scope: Prefect identifies processes by machine hostname and PID. If the infrastructure doesn't support cancellation or if there's a mismatch in the identifier scope, Prefect may not cancel the process effectively. Prefect Cancellation Documentation 2. Manual Process Management: If Prefect's built-in cancellation isn't effective, you might need to manually manage the lifecycle of these shell processes. This could involve writing a custom script to track and terminate these processes when the flow is cancelled. 3. Check for Updates: Ensure you're using the latest version of Prefect as there might be improvements or bug fixes related to process management and cancellation. 4. Community and Issues: It's beneficial to check Prefect's community discussions or GitHub issues for similar problems and potential workarounds. For instance, this GitHub issue discusses related challenges with running jobs. If you're looking for a more integrated solution, you might consider contributing to Prefect's open-source project or raising this issue with the community for potential enhancements in handling such scenarios.