Hello, community 👋
The case is:
I have a project contained several flows. Base environment (whereas prefect server is running, local) contains no extra dependencies (just prefect lib installed). Local agent and docker agent are running.
• Flow A includes f.e. pandas and it runs in docker image (screenshot 1), docker agent assigned
• Flow B includes no extra dependencies
When I’m trying to register Flow A (suppose in base environment). error
No module named 'pandas'
occurred. Flow B is ok.
So the question is: what is the best practice to register flows with different environments and dependencies?
Hope for your support and thank you!
k
Kevin Kho
06/17/2022, 2:22 AM
You will really need the dependencies there, if you really want to avoid installing them, you can move the imports inside a task or you can use
try-except
on the import
g
George Shishorin
06/17/2022, 2:36 AM
Thank you, Kevin. Moving import inside a task works fine.
George Shishorin
06/17/2022, 2:36 AM
George Shishorin
06/17/2022, 2:36 AM
try-except still fails
George Shishorin
06/17/2022, 2:37 AM
Maybe there is more.. accurate or pretty way to register flow? I mean is good practise to register flow on CI/CD using different environment for each flow?
k
Kevin Kho
06/17/2022, 2:55 AM
I don’t think there is much more, it’s either you need the dependencies in the registration environment or your defer the import. Is it an option to pip install the dependencies in your CI/CD script before registration?
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.