https://prefect.io logo
Powered by
Title
g

Gabriel Moran

03/17/2023, 3:35 PM
Hi to everyone, we are migrating from Prefect 1 to Prefect 2. For storage we are using S3 buckets. We were able to store the flows in S3, but when we usa a local agent for debugging it, it seems like it's looking for the files locally instead of getting them from S3. Also, we are getting this error that may be related (but the flow run doesn't stop at this error): 
Flow could not be retrieved from deployment.
I've been testing changing the configuration of the deployment but I had no luck. Permissions to the bucket seem to be ok. do you have any idea of what am I missing? Thanks!
v

Vivek Madenur

03/17/2023, 5:06 PM
Hi Gabriel. try this: 
from prefect.filesystems import S3
s3_storage = S3.load("s3storage") # load the S3 storage object from the block
And you can pass in the storage as below while building the Deployment 
def deploy_flow(flow, name, storage):
    deployment = Deployment.build_from_flow(
        flow=flow,
        name=name,
        storage=storage,
    )
    deployment.apply()
g

Gabriel Moran

03/17/2023, 5:15 PM
that's exactly what I'm doing. But when I run the local agent, instead of using the files stored in S3, it's looking for them locally (I realized that because the paths it wants to use for loading were the local ones and if I change a local filename of a python file containing the flow, and not deploying that change, the agent says that the file was not found)
I checked and the files are stored in S3 correctly
j

James Zhang

03/17/2023, 8:36 PM
can it be it deployed to a remote Prefect if you happened to have one?
#prefect-communityJoin Slack
Powered by