https://prefect.io
Join Slack
whats the correct pattern for spinning our own cloud runner? we simply want to add some initializat...
# ask-community
z
whats the correct pattern for spinning our own cloud runner? we simply want to add some initialization to 
CloudFlowRunner
so that we can initialize sentry and datadog as early as possible, as well as flush our statsd buffer to track metrics. it seems like the place i'd expect it to read from the env var is hardcoded tho https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/kubernetes/agent.py#L562-L586
e.g. im trying to do this
Copy code
class FlowRunner(CloudFlowRunner):  # pragma: no cover
     def run(self, *args, **kwargs):
         <http://logger.info|logger.info>("Flow start")
         try:
             tasks.init()
             result = super().run(*args, **kwargs)
         finally:
             statsd.flush()
         return result
talked briefly to @Kevin Kho about it
cc @Joshua Massover
a
I believe you would, as you said, need to create your custom class and register it via this env variable:
Copy code
PREFECT__ENGINE__FLOW_RUNNER__DEFAULT_CLASS
z
but it looks to be hard coded in that code snip i sent
j
the sentry link is helpful for gathering further metadata regarding the flow and task, thank you! the solution seems like what we've settled on where we're creating an upstream init task before any of our other code runs. It would be nice is if each flow didn't need to worry about "python package level" process initialization.
a
this is not hard-coded afaik, I think you could override it on your run config - LMK if that's not the case and I could open an issue to investigate
k
The agent overrides configs here for the CloudRunner
Maybe you can make you own agent class too haha
z
🤯
think i'll wait for v2 😅
💯 1
upvote 1
k
You’re on the Kubernetes agent right?
z
ye
9 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by