Jaime Raldua Veuthey08/16/2022, 6:49 AM
If no storage is explicitly configured, Prefect will useIf I addstorage by default. Local storage works fine for many local flow run scenarios, especially when testing and getting started. However, due to the inherit lack of portability, many use cases are better served by using remote storage such as S3 or Google Cloud Storage.
I get a copy of all my files on my S3 bucket. But how can I ensure that the Deployment in Prefect Cloud always takes the flow codes from the S3 bucket?
prefect deployment build
Hamza Naanani08/16/2022, 7:53 AM
Jaime Raldua Veuthey08/16/2022, 9:01 AM
After running this command the deployment.yaml will contain "`storage: bucket_path: myS3bucket/folder`". That means that the manifest json file and the python script with the code will be retrieved from this S3 bucket when creating the Deployment and every time the Deployment runs. At this point one can delete the json manifest and the python script with the flow from the local folder since they will not be used (although Prefect creates them both locally and in the bucket anyway, not sure why) To create the deployment using the Prefect CLI:
prefect deployment build ./basic_flow.py:basic_flow -n test -t test --storage-block s3/s3-block-prefect-cloud
This will create the Deployment and since in the yaml file has the S3 as the storage, every time the Deployment runs it will get the flow from the script file in the S3 bucket.
prefect deployment apply deployment.yaml
Hamza Naanani08/16/2022, 10:04 AM