Hey team, , are there any caveats to watch out for...
# ask-community
a
Hey team, , are there any caveats to watch out for if process type worker is deployed in production ? Coz the docs page says this it typically used for getting started / testing purposes. Thanks!
j
Hey there, can you point me to the docs you’re looking at? There might be an opportunity for us to update it
a
Copy code
_description = (
        "Execute flow runs as subprocesses on a worker. Works well for local execution"
        " when first getting started."
    )
prefect/workers/process.py line:135
@jawnsy
This is in code technically
@jawnsy Also, what's the latest stance on this ? Can this be used in production reliably ?
a
Hey @Aditya N, you can use the process worker in production, but there is more responsibility that you’ll need to take on to ensure the worker remains active. For other work types (e.g., Kubernetes) the underlying infrastructure can make sure that the worker stays active and the infrastructure executing flow runs is more separate from the infrastructure running the worker. We recommend running the worker process with a running that will restart the worker process if it fails (e.g. systemd) and running it on infrastructure that can self-heal (e.g. an EC2 auto-scaling group). The process worker works well for local execution because it can be run locally without setting up any other infrastructure.
a
@alex thanks for the response. So I would be running the worker as a pod in an k8s cluster using (https://github.com/PrefectHQ/prefect-helm/blob/main/charts/prefect-worker/values.yaml) configuration to start with.
I guess that should be ok then
@alex For my use-case, I need low latency executions. Pick up flowrun and begin execution (can't wait for infra to get spun up)
Is this a good choice for that ? Since I cant seem to find other alternatives (all of them require additional infra to be spun up)
a
Are the flow runs for your deployment created by a schedule or are they event-driven?
a
@alex Would most likely involve a combination of both
a
Gotcham you can use the
--prefetch-seconds
flag on the
prefect worker start
command to specify the number of seconds in the future that a worker will poll for scheduled flow runs which can help account for the delay in spinning infrastructure. However, that will only work for scheduled flow runs. For ad-hoc flow runs you’ll need to use the process worker in order to get the lowest possible latency.
a
Right so I guess I could go ahead with the
process
worker for now