Mike He
09/16/2022, 9:09 AMcodes
!!! the deployment will overwrite the directory !!! mark down the following 4 credentials for connecting to the FTP server.
FTP_HOST = "127.0.0.1"
FTP_PORT = 21
FTP_USERNAME = "prefect"
FTP_PASSWORD = "--your-password"
• Test codes structure:
codes/
┣ .prefectignore
┣ deploy.py
┗ my_flow.py
• Codes are as follows, please alter the 4 FTP_xxx
parameters with your FTP credentials
# codes/.prefectignore
__pycache__
deploy.py
# codes/deploy.py
from prefect.deployments import Deployment
from prefect.filesystems import RemoteFileSystem
from my_flow import main_flow
FTP_HOST = "127.0.0.1"
FTP_PORT = 21
FTP_USERNAME = "prefect"
FTP_PASSWORD = "--your-password"
ftp_storage_block = RemoteFileSystem(
basepath=f"ftp://{FTP_HOST}/codes",
settings={
"host": FTP_HOST,
"port": FTP_PORT,
"username": FTP_USERNAME,
"password": FTP_PASSWORD,
},
)
ftp_storage_block.save("ftp-localhost", overwrite=True)
deployment = Deployment.build_from_flow(
main_flow, name="Main Flow", storage=ftp_storage_block
)
if __name__ == "__main__":
deployment.apply()
# codes/my_flow.py
from prefect import flow
from prefect.logging import get_run_logger
@flow
def main_flow():
logger = get_run_logger()
<http://logger.info|logger.info>("Hello")
• prefect orion start
• cd codes
and run python deploy.py
• Up an agent: prefect agent start --work-queue "default"
• Run the deployment now with defaults through Prefect Orion UI
Error Messages:
17:15:22.238 | INFO | prefect.agent - Submitting flow run '681f7ee9-ad7c-4961-bebc-6381a954e0b4'
17:15:22.320 | INFO | prefect.infrastructure.process - Opening process 'copper-unicorn'...
17:15:22.326 | INFO | prefect.agent - Completed submission of flow run '681f7ee9-ad7c-4961-bebc-6381a954e0b4'
17:15:25.478 | ERROR | Flow run 'copper-unicorn' - Flow could not be retrieved from deployment.
Traceback (most recent call last):
File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\prefect\engine.py", line 256, in retrieve_flow_then_begin_flow_run
flow = await load_flow_from_flow_run(flow_run, client=client)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\prefect\client.py", line 103, in with_injected_client
return await fn(*args, **kwargs)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\prefect\deployments.py", line 54, in load_flow_from_flow_run
await storage_block.get_directory(from_path=deployment.path, local_path=".")
File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\prefect\filesystems.py", line 296, in get_directory
return self.filesystem.get(from_path, local_path, recursive=True)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\fsspec\spec.py", line 801, in get
self.get_file(rpath, lpath, **kwargs)
File "C:\Users\Mike\AppData\Local\Programs\Python\Python310\lib\site-packages\fsspec\implementations\ftp.py", line 136, in get_file
outfile = open(lpath, "wb")
PermissionError: [Errno 13] Permission denied: 'C:/Users/Mike/AppData/Local/Temp/tmpz2x16tivprefect'
17:15:25.827 | INFO | prefect.infrastructure.process - Process 'copper-unicorn' exited cleanly.
I have also write a post asking this question in Discourse Link.