https://prefect.io logo
Title
a

Austin Weisgrau

03/06/2023, 11:45 PM
prefect deploy build
with an S3 upload block is hanging unless I use
--skip-upload
$ prefect deployment build flows/helloworld/helloworld_flow.py:helloworld -sb s3/prod -n helloworld --skip-upload
Found flow 'Hello World'
Deployment YAML created at '/home/aradox/code/wfp/wfp-prefect/helloworld-deployment.yaml'.

$ prefect deployment build flows/helloworld/helloworld_flow.py:helloworld -sb s3/prod -n helloworld
Found flow 'Hello World'
โœ… 1
I realized it's not actually hanging, but it's slowly uploading the entire repository to the s3 bucket. Is that the intended behavior? If so, why?
I actually still think it hangs, the upload is definitely complete but the method doesn't finish running
j

James Gatter

03/07/2023, 12:53 AM
The command uploads all files/folders starting from the present working directory. Whatever directory you use the command in, you will want to make a .prefectignore file and add to it files you don't want uploaded. https://docs.prefect.io/concepts/deployments/#create-a-deployment-on-the-cli
a

Austin Weisgrau

03/07/2023, 12:54 AM
thank you!
Any idea why it would hang after upload?
j

James Gatter

03/07/2023, 12:55 AM
What output are you seeing from the terminal when the hanging is happening?
Without seeing anything my hunch is that the upload isn't fully complete and it's still working on uploading files that should be ignored
๐Ÿ‘ 1
a

Austin Weisgrau

03/07/2023, 12:58 AM
no output from the terminal
Ok ill try ignoring a bunch of stuff and seeing if that helps
๐Ÿ‘ 1
the output from the terminal is what i posted above - the final command never finishes
r

Ryan Sattler

03/07/2023, 4:17 AM
I saw the same thing, I think itโ€™s just extremely slow at uploading to S3. If I limit the files sent using .prefectignore (eg no libraries or pycache) itโ€™s still slow but finishes after several seconds
๐Ÿ‘ 1
j

James Gatter

03/07/2023, 2:10 PM
Here's my .prefectignore. Maybe there's something you're not catching like .git/ files?
# Project-specific
data/
infrastructure/
utilities/

# General files
.secrets
Dockerfile
*.toml
*.md
*.*sh
*.h5ad
*.parquet
*.png
*.svg

# python artifacts
__pycache__/
*.py[cod]
*$py.class
*.egg-info/
*.egg

# Type checking artifacts
.mypy_cache/
.dmypy.json
dmypy.json
.pyre/

# IPython
profile_default/
ipython_config.py
*.ipynb_checkpoints/*

# Environments
.python-version
.env
.venv
env/
venv/

# MacOS
.DS_Store

# Dask
dask-worker-space/

# Editors
.idea/
.vscode/

# VCS
.*ignore
.git/
.github/
:thank-you: 1