https://prefect.io
Join Slack
Hi All - Relatively new here. Trying to work through a simple example of deploying a prefect project...
# ask-community
m
Hi All - Relatively new here. Trying to work through a simple example of deploying a prefect project that uses s3 storage. Is it common for this method to take 30-60 minutes on a 2 file python project?
c
Hey Mitch - absolutely not normal, sorry you're dealing with that! could you describe where you're running into issues?
m
Sure - I have a entrypoint file that simply imports another python file that for now just contains a function with a 
pass
statement in it. It also has the following deployment definition...
I've been stuck here for around 35 minutes now:
it looks like it's sending data over to the block based on the events firing:
c
hm interesting, i've seen some weird behavior when fsspec and s3fs versions are incompatible; could you try using the block directly to see if that's where the hang is? Something like:
Copy code
s3_bucket_block.put_directory(__file__)
m
looks like it still hangs there...
c
interesting! so in that case, i suggest runing 
pip freeze
and looking specifically at the versions of 
fsspec
and 
s3fs
m
@Chris White it actually appears to be the deployment build that it's getting hung up on. That process basically doesnt finish even from the cli command
Copy code
prefect deployment build -n nightly-symbol-download -p lyric-aws -sb s3-bucket/lyric-s3 flows/nightly_symbol_download.py:nightly_symbol_download
And narrowing it down once more it appears to be the -sb command in the build process. When I omit that, the build happens instantly.
OK I solved this. The problem was using the Block 
s3-bucket
wasn't doing anything... so I found the 
s3
block and it appears to be working. Onto the next step!
c
Sorry I stepped away earlier - huh that’s interesting to hear the bucket block was a dud…
4 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by