Hi everyone, i'm trying to build a pipeline which ...
# ask-community
s
Hi everyone, i'm trying to build a pipeline which have the following components: • Schedule Coordinator: Receive event through a queue and parse the event for scheduling job. We are using github storage to create the flow (to avoid import many dependencies of different workflow into scheduler). • A workpool in K8s • Worker running flow in K8s also Everything is working but we have a problem with the Scheduler Coordinator caused OOM error on K8s even we have tried to increase the memory for the pod, the problem is happened when the schedule coordinator trying to pull github (our github repo size is only 5 MB and we already increase the memory of the pod to 1Gb). Is anyone have this problem before? And beside using github storage to create a flow, is there any different way to create flow without needing to import dependencies of these flows? Thanks!
n
hi - perhaps you can open a discussion that includes some more specifics on what you're doing? one thing i will say is that you don't technically need to have the deps at deployment time, its just that the deployment will not have a verifiable schema for its parameters