kasteph
03/08/2023, 10:37 PMprefect block register -f blocks/gcp.py
for f in deployments/*.py; do
python "$f";
done
Sean Williams
03/08/2023, 10:46 PMkasteph
03/08/2023, 10:47 PMdocker compose exec
on the machine hosting Prefect, then it actually copies it.redsquare
03/08/2023, 11:03 PMkasteph
03/08/2023, 11:55 PMif __name__ == "__main__":
deployment = build_deployment_from_flow(
flow=log,
name="run",
work_queue_name="work",
path=str(Path().resolve()),
entrypoint="deployments/log.py:log",
)
deployment.apply()
and build_deployment_from_flow is a util I wrote:
def build_deployment_from_flow(
flow: str, name: str, work_queue_name: str, path: str, entrypoint: str
):
args = {
"flow": flow,
"name": name,
"work_queue_name": work_queue_name,
}
if os.getenv("DEPLOY_ENV") == "prod":
print("using GCS Block")
return Deployment.build_from_flow(
**args,
path=path + "/deployments",
storage=GCS.load("flows"),
entrypoint=entrypoint.split("/")[-1]
)
return Deployment.build_from_flow(
**args,
**{
"path": path,
"entrypoint": entrypoint,
}
)
redsquare
03/09/2023, 12:14 AMkasteph
03/09/2023, 12:15 AMprefect_flows = GCS(bucket_path="some-temp/prefect-flows/")
prefect_flows.save("flows", overwrite=True)
redsquare
03/09/2023, 12:16 AMprefect block register -f blocks/gcp.py
kasteph
03/09/2023, 12:16 AMredsquare
03/09/2023, 12:17 AMkasteph
03/09/2023, 12:17 AMpython blocks/gcp.py
seems to work fine.redsquare
03/09/2023, 12:17 AMkasteph
03/09/2023, 12:18 AMredsquare
03/09/2023, 12:45 AMkasteph
03/09/2023, 12:45 AMredsquare
03/09/2023, 12:50 AMkasteph
03/09/2023, 12:51 AMredsquare
03/09/2023, 12:54 AMkasteph
03/09/2023, 12:54 AMredsquare
03/09/2023, 12:57 AMdeployment.apply(upload=True)
kasteph
03/09/2023, 1:00 AMI always have my apply in build_from_flow so it auto uploadsOh, what does this mean? Is this just
deployment.apply(upload=True)
you mentioned before?redsquare
03/09/2023, 1:01 AMkasteph
03/09/2023, 1:04 AMredsquare
03/09/2023, 1:05 AMkasteph
03/09/2023, 1:06 AMpython deployments/flow_with_deployment.py
then?redsquare
03/09/2023, 1:07 AMkasteph
03/09/2023, 1:07 AMfor f in deployments/*.py; do
python "$f";
done
redsquare
03/09/2023, 1:08 AMkasteph
03/09/2023, 1:08 AMredsquare
03/09/2023, 1:09 AMkasteph
03/09/2023, 1:12 AMredsquare
03/09/2023, 1:15 AMkasteph
03/09/2023, 1:15 AM