https://prefect.io logo
Channels
Powered by
m

Mitch

07/19/2023, 6:53 PM
Hey all, I'm trying to switch over from using python deployment to the cli so I am able to use github storage. Within my CLI I do not have all of the required dependencies to run my python file as we use a docker image and PyCharm interpreter to run the code. When running the 
prefect deploy
cli command I am getting a module not found error even though the image I included contains that dependency. Is there a way to resolve this?
āœ… 1
j

Jake Kaplan

07/19/2023, 7:43 PM
hey, can you explain what you mean by "image I included contains the dependency"? To run 
prefect deploy
, prefect has to be installed in the environment where you're running the command
m

Mitch

07/19/2023, 7:44 PM
Prefect is installed, however packages such as private pypi package we have (contained in the docker image I have created and stored on aws ECR) are not in the local environment ( powershell ). When running my flows locally, I use the pycharm interpreter as the env which spins up the docker container to "run" the flows.
j

Jake Kaplan

07/19/2023, 7:46 PM
what module isn't found? Is it one of your private packages that's not in your CLI env?
m

Mitch

07/19/2023, 7:46 PM
Yes
Since I don't use my cli env to run flows or python files. I was previously using the pythonic deployment so the flows were being deployed from the docker container
I figured since the flow code is using github as storage, it shouldn't depend on the local file whenI am deploying using 
prefect deploy
, only the prefect.yaml and should fail on task run instead
j

Jake Kaplan

07/19/2023, 7:54 PM
ah I see what you mean
let me check something really quick
šŸ‘ 1
m

Mitch

07/19/2023, 7:55 PM
Yeah it's kinda a unique situation... Is there a good location to see examples of prefect.yaml
j

Jake Kaplan

07/19/2023, 8:09 PM
So at the moment yes, to use the cli command you'll need to be able to load your python file successfully (which includes dependencies)
Two sort of work arounds I have for you: ā€¢ you can use 
client.create_deployment(...)
directly to make the request (
prefect deploy
is a big helper method around that) ā€¢ you can defer your imports so your file can load and you can use the cli
Copy code
# instead of:
import my_private_package

@flow
def my_flow():
   my_private_package.func()

# do this:
@flow
def my_flow():
   import my_private_package
   my_private_package.func()
it is on the backlog to evaluate that kind of case that you just described though
so you don't need to have the code right there for a nice deployment experience
m

Mitch

07/19/2023, 8:13 PM
I'll give it a try. Still determining if we will use agents or workers due to our deployment method
Thanks Jake
šŸ‘ 1
Powered by