Marco Ruta
04/24/2024, 1:19 PMMarvin
04/24/2024, 1:19 PMprefect work-pool create
. Instead, you specify environment variables for a work pool either in the Prefect UI or per-deployment basis.
However, you can override environment variables on a per-deployment basis. Here's a brief outline of how you might do that:
1. You can set environment variables when creating or editing a work pool in the Prefect UI.
2. For deployments, you can use a prefect.yaml
file or the .deploy()
method to override job variables, such as environment variables, for a specific work pool.
If you want to learn more about overriding job variables on work pools, you can refer to the Prefect documentation on Overriding Work Pool Job Variables.
To get a better idea of work pool concepts and job variables, you can check the Work Pools documentation.
If you need to add or edit environment variables for a work pool after it has been created, you will likely need to do so through the Prefect UI or by using programmatic deployments with prefect.yaml
or .deploy()
.Marco Ruta
04/26/2024, 7:49 AMMarvin
04/26/2024, 7:50 AMfrom_source
method. Here's an example of how you could set up a deployment for a flow defined in a remote repository:
python
from prefect import deploy, flow
@flow(log_prints=True)
def local_flow():
print("I'm a flow!")
if __name__ == "__main__":
deploy(
local_flow.to_deployment(name="example-deploy-local-flow"),
flow.from_source(
source="<https://github.com/org/repo.git>",
entrypoint="flows.py:my_flow",
).to_deployment(
name="example-deploy-remote-flow",
),
work_pool_name="my-work-pool",
image="my-registry/my-image:dev",
)
In the provided example, flow.from_source
is used to specify the source of the flow code in a remote repository. You identify the entrypoint, which is the path to the flow function within the remote repository. You can then use to_deployment
to create a deployment with a given name, and select the work pool and image you wish to use.
Please refer to Prefect's documentation on deployments to see more detailed instructions and options for your use case.
Additionally, if you are following a tutorial, you can find relevant information in the work pools tutorial, which briefly touches upon using from_source
with deploy
to use code stored in remote storage such as a GitHub repository.
Keep in mind that you will need to push any changes to the remote flow code to your repository and update the source
argument of from_source
if necessary.