hey all! is there a way to get Prefect to Pickle f...
# prefect-serverc
Christian Nuss
04/12/2022, 11:19 PMhey all! is there a way to get Prefect to Pickle flows including dependencies?
k
Kevin Kho
04/12/2022, 11:21 PM
Not at the moment. Might be possible in the future due to this
cloudpickleKevin Kho
04/12/2022, 11:22 PM
But actually for Orion the goal this quarter is to make packaging easier to do in general whatever that means (pickle or not) such that we can handle more of that for the users
Kevin Kho
04/12/2022, 11:22 PM
We do hear that a lot “why can’t Prefect install my Github repo for me?”
c
Christian Nuss
04/13/2022, 12:46 AMyea i keep hitting walls running containers and Flows that are like
Copy code
from something import thing
with Flow(...):
thing()No module named 'something'somethingpip install -e somethingk
Kevin Kho
04/13/2022, 12:47 AM
Clone and pip install the repo as an ENTRYPOINT of your container
c
Christian Nuss
04/13/2022, 12:49 AMThats a good idea, but a couple downsides i can imagine with that:
• super slow flow startup time for a flow?
• need to inject a SSH key to clone a private repo, somewhere?
Christian Nuss
04/13/2022, 12:52 AMdoes prefect have the ability to augment/customize to the pickling system? could i "tell it what else to pickle" when running register flows?
k
Kevin Kho
04/13/2022, 12:58 AM
Current Prefect does not have that
c
Christian Nuss
04/13/2022, 1:09 AM😢
3 Views