Stephanie Vandenberg
12/29/2023, 12:54 PMprefect.context.get_run_context()
is giving me deployment_id = None
.
This is the way I'm deploying the flow to the cloud:
for config in flows_config:
deployment = Deployment.build_from_flow(
flow=config['flow'],
name=config['name'],
work_queue_name=work_queue_name,
storage=storage,
infrastructure=config['infrastructure'],
tags=config['tags'],
path="",
schedule=config.get('schedule'),
is_schedule_active=config['is_schedule_active']
)
deployments.append(deployment)
for deployment in deployments:
deployment.apply()
Marvin
12/29/2023, 12:55 PMprefect.context.get_run_context()
is designed to provide access to the current run context, which includes information about the flow or task run when you are within a flow or task execution. However, based on the information from the Prefect documentation and a related GitHub pull request, here's what we've found:
1. If get_run_context()
is called when no flow or task run context is available (meaning you are not actively running within a flow or task), the function will raise an exception instead of returning a context with a deployment_id. This suggests that the function should only be used within the appropriate execution context.
2. The deployment_id
might be None
if the function is called outside the context of an active deployment. This might occur if the context from where get_run_context()
is called does not align with a defined deployment object, perhaps during testing or when running flows interactively.
3. A recent pull request has introduced the prefect.runtime
submodule, which provides a user-friendly way to access various pieces of run/deployment configuration dynamically. Within a flow or task function, get_run_context()
might not be the best tool for users to access such information due to the internal nature of the attributes it returns. Instead, users can access the new prefect.runtime
submodule that will return empty values outside of a known context, making it easier to work with such data.
The code snippet you provided for deploying your flow looks correct, but make sure you are calling prefect.context.get_run_context()
within a flow or task that has been initiated through a deployment. If you're doing this outside of a Prefect Cloud/Server-managed flow run, then you may not have a deployment_id available.
If you continue to experience issues with get_run_context()
returning a None
deployment_id during proper flow or task runs, it could be a bug or an issue with how Prefect is managing context. In this case, you may want to investigate further by checking the Prefect GitHub repository or reaching out to the Prefect community for support.Stephanie Vandenberg
12/29/2023, 1:08 PMMarvin
12/29/2023, 1:09 PMdeployment.apply()
method, which you've included in your provided code snippet.
3. Start Flow Run via Deployment: A flow run needs to be initiated from the applied deployment to have an active deployment_id
. This usually means triggering the flow from the Prefect UI, CLI, or an API call, where the deployment has been scheduled or manually kicked off from a work queue associated with the deployment.
4. Ensuring Work Queue Polling: Ensure that your flow runner or agent is polling the correct work queue and is able to pick up the flow run tasks. If the runner/agent does not pick up the tasks, it might seem like there's no active deployment running.
5. Monitoring the Flow Run: Keep an eye on the Prefect UI or logs to monitor the status of your flow run. Successful initiation of a flow run from a deployment will have a deployment_id
associated with it.
6. Use get_run_context()
within a Running Flow: Invoke prefect.context.get_run_context()
within the flow or a task that is definitely running as part of a deployment-triggered run. The context should then include a non-null deployment_id
.
To clarify, an "active deployment" does not mean that there is perpetual activity or an ongoing execution; it means that the deployment has been registered with Prefect Cloud or Server and is capable of initiating flow runs either on a schedule or through manual intervention.
If you've confirmed that all the above steps are correctly followed and your flow is definitely being run as a result of being triggered by an active deployment, but you still see None
for deployment_id
, you may want to check if this is an unexpected behavior or a bug. In this case, consulting the Prefect documentation and community may be necessary to investigate the issue further.Nate
12/29/2023, 3:18 PMStephanie Vandenberg
12/29/2023, 4:39 PMprefect.context.get_run_context().flow_run.deployment_id
prefect.runtime.deployment.id
And both of them are returning NoneNate
12/29/2023, 4:50 PMfrom prefect import flow
import prefect.runtime
@flow(log_prints=True)
def logs_deployment_id():
print(prefect.runtime.deployment.id)
if __name__ == "__main__":
logs_deployment_id.serve(__file__)
10:49:57.386 | INFO | prefect.flow_runs.runner - Runner 'deployment_id_from_runtime' submitting flow run '76eebb35-7609-4f10-9a1c-807671135639'
10:49:57.825 | INFO | prefect.flow_runs.runner - Opening process...
10:49:57.830 | INFO | prefect.flow_runs.runner - Completed submission of flow run '76eebb35-7609-4f10-9a1c-807671135639'
10:49:58.980 | INFO | Flow run 'acrid-bumblebee' - Downloading flow code from storage at '.'
10:49:59.642 | INFO | Flow run 'acrid-bumblebee' - 2bafa3e2-8c82-4c0b-ab4d-250431fd3743
10:50:00.527 | INFO | Flow run 'acrid-bumblebee' - Finished in state Completed()
10:50:01.000 | INFO | prefect.flow_runs.runner - Process for flow run 'acrid-bumblebee' exited cleanly.