@Marvin: I am creating a subflow with

run_deployment

. I would like this subflow to use the same docker image as the parent flow, which I believe I can specify with

job_variables

. Is there a way of accessing the image of the parent flow at runtime?

To achieve what you're asking for – using the same Docker image for a subflow as you do for the parent flow – it sounds like you need dynamic access to the parent flow's image at runtime and pass it as a

job_variable

to

run_deployment

. The excerpts found in the documentation suggest that the image is typically defined statically in a Kubernetes manifest or in some deployment configuration. They do not cover dynamically pulling the image from a currently running flow. However, you might consider the following approach, albeit it's not directly from the documentation: 1. Store the Docker image name as an environment variable or a flow parameter in your parent flow. 2. Access this environment variable or flow parameter in your subflow at runtime. 3. Use the

job_variables

key in your deployment definition to set the Docker image for your subflow to be the same as your parent flow. You can add

job_variables

to your deployment definition like so:

yaml
deployments:
- name: subflow-deployment
  entrypoint: subflow_project/subflow.py:subflow_function
  work_queue_name: your-queue-name
  job_variables:
    env:
      FLOW_CONTAINER_IMAGE: {{ flow_parameter_or_env_variable }}

Replace

flow_parameter_or_env_variable

with the actual parameter or environment variable containing the image name. Remember that you'd need to access the environment variable or parameter in your subflow's logic at runtime and not at build or compile time of the image. For how to do that specifically, you would typically use the

prefect.context

within your flow code to access current flow run parameters or environment variables. Unfortunately, the documentation does not provide an explicit step-by-step process for dynamically obtaining the parent flow's Docker image at runtime and using it in

run_deployment

. For a more precise solution, it might be helpful to refer to community discussions or support forums, or even raise an issue or question on the Prefect GitHub repository or community Slack channel for specific advice. If there's anything specific you'd like to attempt or further information you need, feel free to let me know and I can look deeper—or wallow in the depths of existential despair, whichever seems more useful to you at the time.

@Marvin: can you expand upon what is accessible inside

prefect.context

?

The

prefect.runtime

module contains runtime context that can be accessed within your flows and tasks. This context provides information about the current runtime such as the deployment, flow run, and task run. Here's what each submodule provides: - `deployment`: Access information about the deployment for the current run. This includes aspects such as the deployment's name. - `flow_run`: Access information about the current flow run. Common useful items include run IDs, names, parameters, and state. - `task_run`: Access information about the current task run. This includes the task run name and state. For example, you can access the flow run name and parameters like this:

python
from prefect import flow, task
from prefect import runtime

@flow(log_prints=True)
def my_flow(x):
    print("My name is", runtime.flow_run.name)
    print("I belong to deployment", runtime.deployment.name)
    my_task(2)

@task
def my_task(y):
    print("My name is", runtime.task_run.name)
    print("Flow run parameters:", runtime.flow_run.parameters)

my_flow(1)

Note: As described in the documentation excerpt,

prefect.runtime

is accessed within the flow or task definition and holds attribute information that's simple in type (like

bool

,

int

,

float

,

str

, and

pendulum.DateTime

). For complex types, it's suggested to handle them using other methods, such as mocking them for testing purposes. For more details, you can check the guide on accessing runtime information in the Prefect documentation. It's worth mentioning that you would access this information within the flow logic, so it would require you to code your flow with this context in mind when retrieving the Docker image name for the subflow deployment.