https://prefect.io logo
Title
m

Mike He

09/16/2022, 9:09 AM
Hello everyone. I have a problem about Prefect Agent. It cannot fetch the flow codes from a deployment with FTP RemoteFileSystem Storage Block. Test case are as follows: • Up an FTP server with no directory named
codes
!!! the deployment will overwrite the directory !!! mark down the following 4 credentials for connecting to the FTP server.
FTP_HOST = "127.0.0.1"
FTP_PORT = 21
FTP_USERNAME = "prefect"
FTP_PASSWORD = "--your-password"
• Test codes structure:
codes/
┣ .prefectignore
┣ deploy.py
┗ my_flow.py
• Codes are as follows, please alter the 4
FTP_xxx
parameters with your FTP credentials
# codes/.prefectignore
__pycache__
deploy.py


# codes/deploy.py
from prefect.deployments import Deployment
from prefect.filesystems import RemoteFileSystem

from my_flow import main_flow

FTP_HOST = "127.0.0.1"
FTP_PORT = 21
FTP_USERNAME = "prefect"
FTP_PASSWORD = "--your-password"

ftp_storage_block = RemoteFileSystem(
    basepath=f"ftp://{FTP_HOST}/codes",
    settings={
        "host": FTP_HOST,
        "port": FTP_PORT,
        "username": FTP_USERNAME,
        "password": FTP_PASSWORD,
    },
)
ftp_storage_block.save("ftp-localhost", overwrite=True)
deployment = Deployment.build_from_flow(
    main_flow, name="Main Flow", storage=ftp_storage_block
)

if __name__ == "__main__":
    deployment.apply()


# codes/my_flow.py
from prefect import flow
from prefect.logging import get_run_logger

@flow
def main_flow():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Hello")
prefect orion start
cd codes
and run
python deploy.py
• Up an agent:
prefect agent start --work-queue "default"
• Run the deployment now with defaults through Prefect Orion UI Error Messages:
17:15:22.238 | INFO    | prefect.agent - Submitting flow run '681f7ee9-ad7c-4961-bebc-6381a954e0b4'
17:15:22.320 | INFO    | prefect.infrastructure.process - Opening process 'copper-unicorn'...
17:15:22.326 | INFO    | prefect.agent - Completed submission of flow run '681f7ee9-ad7c-4961-bebc-6381a954e0b4'
17:15:25.478 | ERROR   | Flow run 'copper-unicorn' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\prefect\engine.py", line 256, in retrieve_flow_then_begin_flow_run
    flow = await load_flow_from_flow_run(flow_run, client=client)
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\prefect\client.py", line 103, in with_injected_client
    return await fn(*args, **kwargs)
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\prefect\deployments.py", line 54, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=deployment.path, local_path=".")
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\prefect\filesystems.py", line 296, in get_directory     
    return self.filesystem.get(from_path, local_path, recursive=True)
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\fsspec\spec.py", line 801, in get
    self.get_file(rpath, lpath, **kwargs)
  File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\fsspec\implementations\ftp.py", line 136, in get_file   
    outfile = open(lpath, "wb")
PermissionError: [Errno 13] Permission denied: 'C:/Users/Mike/AppData/Local/Temp/tmpz2x16tivprefect'
17:15:25.827 | INFO    | prefect.infrastructure.process - Process 'copper-unicorn' exited cleanly.
I have also write a post asking this question in Discourse Link.
👍 1