https://prefect.io logo
#prefect-community
Title
# prefect-community
a

Albert Wong

03/04/2023, 10:37 AM
I take it that you need a remote filesystem block to deploy flows remotely? When I deploy with filestorage, the remote prefect server is trying to load a file path that looks like a path in my local system. I have github, but haven't been able to figure out why the deploy does not upload files properly. I get the following warning:
Copy code
Deployment storage GitHub(repository='<'redacted'>, reference=None, 
access_token=SecretStr('**********'), include_git_objects=True) does not have upload capabilities; no files uploaded.  Pass --skip-upload to suppress 
this warning.
This is after I execute the following deploy command:
Copy code
prefect deployment build -n sync_repository_tables_flow -q default flows/sync_repository_tables_flow.py:sync_repository_tables_flow -a -sb github/prod-gbi-github-dataflowdeployments
Anyone know what I'm missing?
c

Christopher Boyd

03/06/2023, 2:40 PM
github is read only (
get-directory
is listed on the block) to retrieve storage, and is not a valid remote file store for pushing content
if you’re using a docker image, you can pass the path and entrypoint - otherwise, you’d need a file store that supports put-directory / write-path like azure, s3, gcs
a

Albert Wong

03/06/2023, 4:41 PM
Thanks @Christopher Boyd, this is compelling, and I'll take a look. Can I also manage different versions of python + libs through this method?
Hmm, looks like the docker infrastructure block cannot find the flow either. Do you mean I should build a custom image with the flows in the image? If so, is there any documentation on how to do that?
for reference, this is what I get when running the flow
Copy code
02:48:58.664 | INFO    | Flow run 'rare-rook' - Downloading flow code from storage at '/opt/prefect/flows'
02:48:58.668 | ERROR   | Flow run 'rare-rook' - Flow could not be retrieved from deployment.
actually, if github is readonly, can I just manually check in the flows into github for retrieval on the remote server? i.e. I deploy with the github storage block defined, check in the flow code, and then run?
oh shoot, that works! haha. There's a demo video here on the subject:

https://www.youtube.com/watch?v=Nv1eD0VDedA

w

Will Truong

03/17/2023, 4:28 AM
Thank you Albert, I am currently trying to implement this too.