@Marvin Using prefect 3.x, is it possible to chain the execution of two flow using an automation and share a parameter from the source flow (the one that triggers the event to run the destination flow) to the destination flow ? If so, show a simple example on how to do this

Yes, in Prefect 3.x, it's possible to chain the execution of two flows using automation, and you can share a parameter from the source flow to the destination flow. You can use event triggers or custom triggers to orchestrate such execution chains, ensuring that one flow runs upon the successful completion of another. Additionally, shared context or parameters can facilitate inter-flow communication. ### Example Here's a simple conceptual overview of how you might set it up: 1. Define the Source Flow: Define your source flow, which will execute first and contain the parameter you want to pass. 2. Event/Custom Trigger: Set up an automation using an event or custom trigger that listens for the completion of the source flow. 3. Define the Destination Flow: Define your destination flow, which will execute based on the trigger and receive the parameter from the source flow. 4. Parameter Sharing: You can share parameters by using task results from the source flow as inputs for tasks in the destination flow. This can be done through flow results or by deploying both flows together and managing parameters programmatically. ### References 1. Prefect Event Triggers Documentation - Learn about setting up triggers to automate flow executions. 2. ControlFlow and Task Context - Understand how to use flows in ControlFlow to manage shared contexts and parameters between tasks and flows. By using Prefect's automation and ControlFlow's structured approach for managing flow contexts, you can streamline and automate complex workflows gracefully. It's like giving your workflows a little dance routine to follow!

@Marvin Can this share of parameters be configured on the automation side? or it is only possible programatically by running nested flows in a single flow ?

In Prefect 3.x, sharing parameters between flows is typically handled programmatically, especially if you are running nested flows within a single flow. The automation side of Prefect mainly focuses on triggering executions through event triggers and does not inherently support configuring parameter sharing directly within automations. ### Key Points: 1. Programmatic Parameter Sharing: You generally need to pass parameters programmatically when one flow initiates another. This can be accomplished by creating a nested flow structure where the parent flow passes parameters to the child flows. 2. Using Prefect API: Parameters can be specified and passed through the Prefect API when defining deployments. However, these parameters need to be set up when creating or deploying the flows and cannot be dynamically altered directly within an automation. 3. Automations: Automations in Prefect focus on defining when and how to trigger flows using event-based triggers, but they don’t modify or handle parameter passing—it’s more about execution logic rather than data manipulation. For more information, you can check these resources: - Prefect Automations & Triggers Documentation - See how automations work. - Nested Flows and Parameter Handling - Understand more about handling flows and parameters programmatically. In summary, while Prefect automations can trigger flows, parameter sharing needs to be coordinated within the flow setup or via the API. I suppose automations need to hit the library for their social skills.

@Marvin does prefect 3.x has the capability to trigger events based on external triggers like createObject events on an S3 bucket ?

In Prefect 3.x, there is support for custom event triggers, which can be set up to respond to various external events, though specific native support for S3 "createObject" events isn't explicitly outlined in the standard documentation. However, Prefect allows for integrations with other systems and can generally be configured to respond to external triggers like those from S3 by using a combination of event listeners or webhooks that pass data to Prefect-specific endpoints, thus initiating flows. Here's a conceptual approach you might consider for integrating such functionality: 1. S3 Event Setup: Configure your S3 bucket to send notifications to an AWS Lambda function, SNS topic, or SQS queue when an object is created. 2. Trigger with Webhooks or an API: Use a webhook or AWS service (like Lambda) to send an HTTP request to trigger a Prefect flow via its API. This action can be set up to include necessary details to trigger specific flows and pass parameters. 3. Prefect Flow Configuration: In Prefect, create a flow that listens for these webhook calls and starts execution based on the HTTP request data. For detailed setup and custom event triggers, you might want to check resources or community plugins related to integrating AWS services with Prefect or consult the Prefect Events Documentation. So, in essence, yes, it’s possible! Though, you might need a bit of creativity and some AWS glue (not the literal AWS Glue service; just metaphorical).