Hi I am on prefect 2.0 cloud and my deployment sud...
# prefect-clouds
Siva Balusu
01/07/2023, 3:10 AMHi I am on prefect 2.0 cloud and my deployment suddenly stopped working with the below error. Deployment pushed the flows to s3 bucket but agent is having issues to get those flows?
Copy code
Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/prefect/engine.py", line 260, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "/usr/local/lib/python3.10/site-packages/prefect/client/utilities.py", line 47, in with_injected_client
return await fn(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/prefect/deployments.py", line 167, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 463, in get_directory
return await self.filesystem.get_directory(
File "/usr/local/lib/python3.10/site-packages/prefect/filesystems.py", line 308, in get_directory
return self.filesystem.get(from_path, local_path, recursive=True)
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 113, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 98, in sync
raise return_result
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 53, in _runner
result[0] = await coro
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 551, in _get
rpaths = await self._expand_path(rpath, recursive=recursive)
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 751, in _expand_path
out = await self._expand_path([path], recursive, maxdepth)
File "/usr/local/lib/python3.10/site-packages/fsspec/asyn.py", line 775, in _expand_path
raise FileNotFoundError(path)
FileNotFoundError: ['prefect-bucket/prefect_test2']a
Anna Geller
01/08/2023, 8:38 PM
this must be related to the threads below and should now work after upgrading to 2.7.7 - could you confirm?
Anna Geller
01/08/2023, 8:39 PM
also next time, could you add code and related follow up to the thread? helps to keep main channel clean
s
Siva Balusu
01/08/2023, 10:00 PM Copy code
from start_aggregration_process import start_aggregation_process
from prefect.deployments import Deployment
from prefect.filesystems import S3
deployment = Deployment.build_from_flow(
flow=start_aggregation_process,
name="pipeline-V2.0",
version="2",
tags=["DATA_PIPELINE"],
storage=S3.load("myawsblock")
)
if __name__ == "__main__":
deployment.apply()Siva Balusu
01/08/2023, 10:12 PM@Anna Geller Thanks for the suggestion. I have upgraded the prefect agent and my local version but still no luck. The same code block worked until yesterday.
Siva Balusu
01/09/2023, 2:17 AM@Anna Geller Removing prefect-aws, reinstalling s3fs, downgrading prefect-snowflake to 0.2.2 did the trick. I am able to execute the deployment now.
a
Anna Geller
01/09/2023, 10:50 AM
Nice work! Thank you for the update 🙌
3 Views