Hi guys. When I build and apply a deployment I'm g...
# ask-communityt
Tolga Karahan
02/15/2023, 10:05 AMHi guys. When I build and apply a deployment I'm getting error below. Storage block is not set, thus local storage is used. Infra block specifies k8s cluster and an agent works in there, so basically I want to run a flow from my local storage in k8s cluster specified by infra block. Any idea?:
✅ 1
r
redsquare
02/15/2023, 10:22 AMlocal storage is not supported for Kubernetes Jobs
🙌 1
t
Tolga Karahan
02/15/2023, 1:29 PM@redsquare thanks
Tolga Karahan
02/15/2023, 1:29 PMIs docker storage supported for k8s jobs to pull image from a registry and run as a job?
r
redsquare
02/15/2023, 1:50 PMInteresting - perhaps you could fool it by setting ignore storage and setting the path to the location in the docker image https://github.com/PrefectHQ/prefect/blob/main/src/prefect/deployments.py#LL156C9-L156C27
t
Tolga Karahan
02/15/2023, 2:27 PM@redsquare so Docker storage is not supported anymore by default?
r
redsquare
02/15/2023, 2:29 PMit is but not sure if you use your own docker image within a k8s job->pod - does it expect the storage to come remotely
t
Tolga Karahan
02/15/2023, 2:31 PMyeah it was how we use Prefect 1
r
redsquare
02/15/2023, 2:33 PMlooks like it will be find if you set ignore_storage - will just use the path for the flow
t
Tolga Karahan
02/15/2023, 2:36 PMokay, thanks @redsquare 🙏
r
redsquare
02/15/2023, 2:38 PMwonder if you could set that, attach a volume by a patch in a customization for the k8s job it would work
👍 1
t
Tolga Karahan
02/16/2023, 2:29 PMI just wonder if flow code doesn't come from remote, but embedded in builtin image. May I use this image as the image for infra block to execute flows? Thanks in advance. 🙏
r
redsquare
02/16/2023, 2:34 PMI have not tried but see no reason for a limitation here to allow you to run a custom docker image in a k8s job and reading the flow from the image
redsquare
02/16/2023, 2:36 PMIf you set the ignore storage to true
redsquare
02/16/2023, 2:42 PMyou should be able to store the flows on a volume in k8s and have the pod mount the volume+exxecute the flow
t
Tolga Karahan
02/16/2023, 2:44 PMI see thank you, we'll give it a try, our main issue is we have quite heavy initialization so we want to use our custom image
r
redsquare
02/16/2023, 2:49 PMfeels a pain to build and publish a new image everytime a flow changes
redsquare
02/16/2023, 2:50 PMor you add a new one
t
Tolga Karahan
02/16/2023, 4:07 PMyeah
Tolga Karahan
02/16/2023, 4:08 PMso it looks like using custom image as execution environment but getting flows from some storage is better
r
redsquare
02/16/2023, 4:08 PM100%
t
Tolga Karahan
02/16/2023, 4:09 PMI'll try using custom image as infra block image, and also a storage maybe s3 or remote git repo
r
redsquare
02/16/2023, 4:09 PMwe use k8s with s3 storage - works great
t
Tolga Karahan
02/16/2023, 4:12 PMyour pods are also using a custom image?
Tolga Karahan
02/16/2023, 4:12 PMor prefects own image
r
redsquare
02/16/2023, 4:13 PMcurrently prefects own until we are sure about common requirements across all flows - then we will create a custom image - but only when that is clear
redsquare
02/16/2023, 4:14 PMtime for the flow to run is not uber critical for us currently - so we have lots of pip installs
redsquare
02/16/2023, 4:14 PMlots = 5-15
t
Tolga Karahan
02/16/2023, 4:14 PMI see, sounds good
Tolga Karahan
02/16/2023, 4:14 PMML dependencies are a bit problem, libraries are heavy
r
redsquare
02/16/2023, 4:15 PMyeah, we are light on ML but use MindsDB on top of Clickhouse currently
t
Tolga Karahan
02/16/2023, 4:15 PMoh interesting
Tolga Karahan
02/16/2023, 4:16 PMI know it but didn't find any occasion to use 🙂
r
redsquare
02/16/2023, 4:16 PMclick or minds:)
t
Tolga Karahan
02/16/2023, 4:17 PMMindsDB, have no idea on click 😄
r
redsquare
02/16/2023, 4:20 PMThey are both great, clickhouse especially
t
Tolga Karahan
02/16/2023, 4:22 PMyeah, mindsdb look very interesting, I'll take a look to other