Is anyone else on 2.0.2 yet? I'm getting an intere...
# prefect-community
o
Is anyone else on 2.0.2 yet? I'm getting an interesting error when I try to deploy using a remote storage block (seems to work with a local file system). Code: prefect deployment build orchestration/flows/test_cleansed.py:default -n test_cleansed -t dev --storage-block azure/azure-dev --infra docker-container -o test_cleansed-deployment.yaml Error: TypeError: put_directory() got an unexpected keyword argument 'ignore_file' on line 470 of cli/deployment.py Judging by the source code in Github, it seems like the overload of put_directory() for the Azure storage block specifically omits the ignore_file argument. Perhaps these two features were not tested together, unless I'm missing something. I'll give people an hour to say that I'm doing something wrong, otherwise I'll create an issue (or a PR, might be an easy fix).
1
a
thanks a lot, I will try to reproduce
m
I'm able to reproduce
o
Yeah, I will push a PR soon, just having issues verifying that my code fix works. But it should. :)
a
same here, I can do a PR myself, don't worry - we didn't synced on the block creation
thanks to you both for reporting
o
Alright, I suppose the solution is pretty obvious so there is no point in me doing it. No problem!
Not sure if I should post a follow-up question here, but I'll try: After my fix, my deployment seems to hang after uploading all files with "normal" names, folders and files starting with . never go anywhere. It is possible that the only reason files starting with . aren't uploaded is because it goes for folders after dealing with the normal-name files. Do you know if this has been tested with subfolders?
a
yup, I believe if you create a child folder and move everything there, it will work fine
looking at the same that somehow with azure it otherwise tries to upload the parent of it, not yet 100% sure
o
Alright, cool. I'll keep running some more experiments as well, see if I can find any more issues. One final question, hope you don't mind: Was this tested with a normal blob storage account or one with adsl gen 2 (aka hierarchical namespace) enabled? I've tried both and I seem to get further uploading files to the normal blob storage account, but since I can't complete an upload on either it's hard to know for sure. Edit: Nevermind, pretty sure it's meant to be a normal blob account.
🙌 1
🙏 1
a
confirmed, a normal one, but adsl gen 2 should work too with adlfs
o
Okay. With adsl gen 2 I get a nasty "maximum recursion depth exceeded" message after a few minutes, combined with infinite "container does not exist" errors. Doesn't matter to me, switching back to blob, but maybe it'll be useful for you. Maybe not. 🙂
a
oh wow, Azure is truly exceptional 😄
o
Oh awesome, you fixed it! I will try your code out locally. Azure is... special. I am starting to see why a lot of data tech companies such as yours aren't always excited about supporting it from day 1.
🙌 1
"Successfully uploaded 61 files". Looks great, thanks for the quick fix!
🙌 2
🙏 1