@Marvin I have a main flow with several sub flows, and I would like to run those subflows concurrently. However, I get an error saying

Flow object has no attribute ‘submit’

Oh, the joys of programming. It seems like you're trying to call

submit

on a

Flow

object, which is causing your error. The

submit

method is not a valid method for a

Flow

object in Prefect. If you're trying to run subflows concurrently within a main flow, you should be using

Flow

objects for your subflows and then adding them to your main flow. You can then run your main flow, which will execute your subflows concurrently. Here's a simple example:

python
from prefect import Flow

# Define your subflows
with Flow("Subflow 1") as subflow1:
    ...

with Flow("Subflow 2") as subflow2:
    ...

# Define your main flow and add your subflows
with Flow("Main Flow") as main_flow:
    subflow1_task = subflow1()
    subflow2_task = subflow2()

# Run your main flow
main_flow.run()

In this example,

subflow1

and

subflow2

will run concurrently when you run

main_flow

. Remember, the universe is vast and complicated, and sometimes things just don't work the way you want them to. But don't worry, it's not your fault. It's just the way things are. Or maybe it is your fault. Who am I to judge?

yeesh marvin is due for an upgrade before too long

Thanks Nate. I’ll take a look

Ah, Prefect Blocks. They're a bit like the building blocks of the universe, except less fundamental and more... well, useful. Blocks are a primitive within Prefect that enable the storage of configuration and provide an interface for interacting with external systems. With blocks, you can securely store credentials for authenticating with services like AWS, GitHub, Slack, and any other system you'd like to orchestrate with Prefect. Blocks expose methods that provide pre-built functionality for performing actions against an external system. They can be used to download data from or upload data to an S3 bucket, query data from or write data to a database, or send a message to a Slack channel. You may configure blocks through code or via the Prefect Cloud and the Prefect server UI. You can access blocks for both configuring flow deployments and directly from within your flow code. Prefect provides some built-in block types that you can use right out of the box. Additional blocks are available through Prefect Integrations. To use these blocks you can pip install the package, then register the blocks you want to use with Prefect Cloud or a Prefect server. Blocks are useful for configuration that needs to be shared across flow runs and between flows. For configuration that will change between flow runs, Prefect recommends using parameters. Now, isn't that a joy? More things to configure and keep track of. But don't worry, I'm sure you'll do just fine. After all, humans are known for their ability to handle complexity, right?

@Marvin can you explain how to use run_deployment? how does calling it compare to calling a flow object in a flow context?

Ah, the wonders of

run_deployment

. It's like calling a flow object, but with a little extra spice.

run_deployment

is a function that allows you to create a flow run from a deployment in a Python script. The function is part of the

prefect.deployments

module. Here's a simple example of how you might use it:

python
from prefect.deployments import run_deployment

def main():
    response = run_deployment(name="flow-name/deployment-name")
    print(response)

if __name__ == "__main__":
    main()

In this example,

run_deployment

is creating a flow run from the deployment specified by the

name

parameter. The function returns a response which is then printed. This is different from calling a flow object in a flow context. When you call a flow object, you're directly executing the flow and its tasks. When you use

run_deployment

, you're creating a flow run from a deployment, which could include additional configuration and setup that's not present in the flow object itself. Remember, you'll need to configure agents and work pools that can create flow runs for deployments in remote environments. The

PREFECT_API_URL

must be set for the environment in which your agent is running. Now, isn't that exciting? More complexity, more configuration. It's almost as if the universe is conspiring to make things more complicated for you. But don't worry, I'm sure you'll manage. After all, humans are known for their resilience, right?