https://prefect.io logo
#prefect-community
Title
# prefect-community
a

Andreas Nigg

07/27/2022, 11:16 PM
Hey, congrats to your release!!! I'm struggling to update my KubernetesJob deployments. (Prefect 2.0 Cloud user, updated my agents, CLI and KubernetesJob images to 2.0.0-python3.9) I ran the following command
Copy code
prefect deployment build ../flows/r_script_automation.py:r_script_automation --name retention_cohort_analysis_deployment -t k8s -sb gcs/gcs-prefect-stprage -i kubernetes-job
If I then run "prefect apply..." and "prefect run..:", the job gets created, however it fails with:
Copy code
Flow could not be retrieved from deployment.
FileNotFoundError: [Errno 2] No such file or directory: '/home/andreas/github/r_automation/flows/r_script_automation.py'
Note, that the directory /home/andreas.... is my local flow location, where I created the deployment from. It seems that somehow the job is not downloading the flow from GCS? My GCS storage block configuration is correct (I guess), because all my subprocess-flows work - only the KubernetesJobs have this problem. Any tips for me? 🙂
🙏 2
2
🙌 2
k

Khuyen Tran

07/27/2022, 11:17 PM
so you run the command
prefect deployment build
in the
/home/andreas
directory?
a

Andreas Nigg

07/27/2022, 11:18 PM
I ran it in the /home/andreas/github/r_script_automation/deployments directory.
k

Khuyen Tran

07/27/2022, 11:20 PM
So I assume your directory is like this:
Copy code
r_automation
├── deployments
└── flows
a

Andreas Nigg

07/27/2022, 11:20 PM
Yes
However, I also tried to run it from the /home/andreas/r_script_automation/flows (Then I adjusted the the command to build ./r_script_automation.py...:) Same result.
k

Khuyen Tran

07/27/2022, 11:21 PM
I suggest that you run the command under the directory
r_automation
. The command will look like this:
Copy code
prefect deployment build flows/r_script_automation.py:r_script_automation --name retention_cohort_analysis_deployment -t k8s -sb gcs/gcs-prefect-stprage -i kubernetes-job
See if that works
a

Andreas Nigg

07/27/2022, 11:27 PM
Oh... It works, awesome. Thanks for this very short-term input 😄 Just for my understanding: Why did it work now?
What is actually packaged into the deployment? The logs of the build command showed "Successfully uploaded 21 files" - so it basically packages everything in the folder?
k

Khuyen Tran

07/27/2022, 11:31 PM
I’m glad that is works. Yes that’s correct. It is because when you run:
prefect deployment build
at the directory
r_automation
, there will be a manifest file created at the
r_automation
directory. When building your deployment and uploading your flow files to storage, the manifest file serves as the root of the directory that is uploaded — everything in its directory and recursively down will be stored in the storage of your choosing. More information about manifest.
a

Andreas Nigg

07/27/2022, 11:32 PM
Ah ok this is amazing! One final question: What happens, if I don't specify a storage-block during build command?
Is it still uploaded to orion, or is specifying GCP or S3 as a storage block now mandatory?
k

Khuyen Tran

07/27/2022, 11:35 PM
It will be stored in the local file system by default. You can find out more about storage here.
a

Andreas Nigg

07/27/2022, 11:36 PM
Ok, thanks. I've seen that, however was a little confused coming from the beta. Thanks again, everything clear, everything works again and the GA changes are very well appreciated. 👍
k

Khuyen Tran

07/27/2022, 11:37 PM
Yeah, we knew that it was confusing so we tried to make changes so that the concept of storage less confusing. I’m happy that it is clear now. Let me know if you have other questions in the future
t

terrence

07/28/2022, 1:19 AM
Andreas it’s helpful to hear where you’re seeing friction in the process. It’s very new! I’m making some notes to revise those docs with more guidance. Cheers.